Building agents that survive their own execution

For about nine hours one night, our agents stopped moving. Nothing crashed. Queued work just sat there.

The trigger was mundane. During an RDS restart, a transient database failure surfaced inside Conol's worker polling loop. The loop stopped draining the queue, but the process stayed alive because it still held open handles. No supervisor replaced it. The process sat there, breathing, doing nothing.

What saved us was not the loop. It was what sat underneath it. Every piece of agent state and every effect the agents had emitted was durable in Postgres. When we restarted the worker, queued work resumed from where it had stopped. We did not reconstruct user sessions or replay conversations from logs.

That night is the reason this post exists. The gap between "the model is smart" and "the system is reliable" is not filled by a better prompt. It is filled by treating an agent turn as durable data instead of a call stack.

Here is where most agent runtimes start:

async function runAgent(input: string) {
  const messages = [{ role: "user", content: input }]

  while (true) {
    const response = await llm.chat(messages)
    messages.push(response)

    if (!response.toolCalls?.length) return response.content

    for (const call of response.toolCalls) {
      const result = await runTool(call)
      messages.push({ role: "tool", content: result })
    }
  }
}

This is fine on a laptop. In production it has a fatal property: all important state lives on the stack and in local variables. The message history, loop position, and half-finished tool call exist only inside one process. If that process dies or hangs, the turn goes with it.

The model can be as intelligent as we like. The runtime around it is only as reliable as the weakest process holding its stack.

The mental model that helped us most came from an old corner of programming-language theory: algebraic effects. We do not implement them. But the way they separate "what to do" from "how to do it" is exactly the separation a durable runtime needs.

Algebraic effects in five minutes

Start with a plain question: when code needs something from the outside world, how does it ask?

A computational effect is anything a function does beyond returning a value. Reading input, writing a file, calling a model, mutating state, and throwing an exception are all effects.

Algebraic effects let code name those requests without deciding how they are answered. The program is split into two roles:

• An operation describes the request.
• A handler decides what that request means.

The function performing an operation does not know how it will be serviced.

function greet() {
  let name = perform AskUser("What is your name?")
  print("Hello, " + name)
}

handle greet() with {
  AskUser(question, k) => resume k("Ada")
}

perform AskUser(...) does not read from a terminal, web form, or test fixture. It performs an operation and expects a string back.

The handler receives the operation and a continuation k. The continuation represents the rest of the computation from the point where the operation occurred. Here it is roughly:

k(name) = print("Hello, " + name)

The handler decides what happens next. Depending on the language and implementation, it may resume once with a value, never resume — abandoning the remaining computation like an exception — or resume several times to explore multiple branches, supporting nondeterminism and backtracking.

The useful intuition is a resumable exception. A normal exception unwinds the stack and never comes back. An effect operation may transfer control to a handler and later resume at the exact point that asked.

Real implementations constrain this. OCaml 5 uses one-shot continuations: each captured continuation may be resumed at most once. Multi-shot continuations are more expressive but costlier to implement safely.

Effect handlers do not automatically solve every composition problem associated with monads. They are a different abstraction. The part that matters for Conol is simpler: the computation declares what it needs, while an interpreter elsewhere decides how to satisfy it.

From programming-language theory to practical handlers

The lineage did not begin with one paper.

Eugenio Moggi introduced monads as a structuring principle for computational effects in 1991. Gordon Plotkin and John Power developed an algebraic account of effects in the early 2000s, describing operations through equations. Plotkin and Matija Pretnar formalized effect handlers in 2009, turning those operations into something programs could interpret through continuations.

The word "algebraic" is literal. Operations obey laws. For state, for example:

set(x); set(y) = set(y)

Writing x and immediately writing y has the same final effect as writing only y. Such equations describe an effect independently of its implementation.

Effect handlers now have practical use in OCaml 5, particularly for direct-style concurrency and asynchronous I/O. Libraries such as Eio let application code look synchronous while effect handlers implement scheduling underneath. Research languages such as Koka, Eff, and Effekt explore typed effects and handlers for state, exceptions, generators, backtracking, resource handling, and test interpreters.

Some mainstream systems have a similar shape without being algebraic-effect systems. React Suspense routes suspension to a boundary, and durable workflow engines journal or replay external calls. These are useful analogies, but they do not provide the same continuation and effect-type machinery.

Conol takes the separation, not the machinery

Our thesis is straightforward. We do not capture continuations. We do not have an effect type system. What we take from algebraic effects is the separation, and we implement it with durable data.

• An operation becomes a durable effect value written to Postgres.
• A handler becomes a worker that reads and interprets effects.
• Resumption means reconstructing persistent state and continuing, not restoring a suspended stack.

An agent turn should not depend on one process staying alive. It should live as a sequence of effects and state transitions that any worker can continue.

The rest of the design follows from three patterns.

Pattern 1: Actions become effects

The first move is to stop performing external work directly on the agent's call stack. An effect is a tagged value:

export type Effect =
  | { type: "user"; sessionId: string; prompt: UserMessage[]; visibleAt?: Date }
  | { type: "llm"; sessionId: string; visibleAt?: Date }
  | { type: "tool_call"; sessionId: string; toolCall: ToolCall; visibleAt?: Date }
  | { type: "reply"; sessionId: string; toolCallId: string; reply: ContentBlock[] }
  | { type: "timer"; sessionId: string; id: string; trigger: TimerTrigger; visibleAt?: Date }

There is no callLLM hidden inside a closure. There is a value such as { type: "llm", sessionId }, meaning that a session owes an LLM turn.

A registry maps effect types to handlers:

export type Handler<T extends EffectType> =
  (context: HandlerContext<T>) => Promise<MutationResult>

export type HandlerRegistry = {
  [K in EffectType]?: Handler<K>
}

The queue is inspectable. We can see what a session is waiting for. The worker interpreting an effect need not be the process that emitted it. Tests can replace real handlers with deterministic ones. The runtime can defer work without keeping a process asleep.

Two fields carry much of the scheduling model. visibleAt says when an effect may next be claimed — backoff and timers use the same queue primitive. An optional stable id gives repeatable work a logical identity, so dispatching the same timer ID replaces its queued row rather than creating another.

A pending effect is a fact we can store. A pending function call is not.

Pattern 2: State advances through pure durable mutations

Effects describe work. Mutations advance state.

export type MutationResult =
  | { behavior: undefined; state: AgentState; ops?: Effect[] }
  | { behavior: "drop" }
  | { behavior: "replay"; visibleAt?: Date }

export type Mutation =
  (sessionId: string, from: AgentState) => MutationResult

A successful mutation returns the next state and the effects created by that transition. drop means the input is stale, duplicated, or already handled. replay returns the claimed effect to the queue, optionally at a future time.

Every committed state has a monotonic version. Writes pass through a compare-and-swap loop:

async function mutate(mutation, store, sessionId, was) {
  let now = was

  while (true) {
    const result = mutation(sessionId, now)
    if (result.behavior !== undefined) return result

    const next = { ...result.state, version: now.version + 1 }
    const current = await store.cmpxchg(
      sessionId,
      now.version,
      next,
      result.ops ?? [],
    )

    if (!current) return { ...result, state: next, ops: undefined }
    now = current
  }
}

If the version still matches, the write commits. If another writer moved the state first, cmpxchg returns the current state and the runtime reruns the mutation against it.

Purity makes this rebase safe. A mutation may be evaluated several times, but only the winning result is committed. External I/O cannot happen inside the mutation because a CAS retry could duplicate it. Follow-up work returns as ops and runs later as effects.

Pattern 3: Commit state and consequences together

The two halves must move together. When a handler services an effect, its state change and newly emitted effects must become durable atomically.

If state advances but the effects disappear, follow-up work is lost. If the effects commit but state does not, the runtime may execute work the agent does not know it requested.

Conol writes both in one Postgres transaction using the transactional outbox pattern:

BEGIN;

UPDATE agent_sessions
SET state = $new_state,
    state_version = $next_version
WHERE id = $session_id
  AND state_version = $expected_version;

INSERT INTO effect_queue (id, session_id, effect, visible_at)
VALUES (...);

COMMIT;

If the transaction commits, state and follow-up effects become durable together. If it rolls back, neither happens.

Workers pull from the queue with a lease. Claiming an effect stamps it with a claim_token and moves visible_at into the future for the lease duration. Acknowledgements, retries, and lease extensions require the token to match.

This provides at-least-once delivery. A worker may crash after starting work but before recording it, so the effect can be delivered again. Result mutations therefore assume redelivery:

• A tool result is write-once by toolCallId.
• LLM stream events are tied to an llmCall.id; stale events are dropped.
• Reusing an effect ID replaces the queued row instead of duplicating it.

One tool-using turn, five durable commits

A single tool-using turn becomes a sequence of durable commits:

1. A user effect arrives. Commit the message and emit an llm effect.
2. The llm handler runs the model. Commit the assistant message and emit a tool_call effect.
3. The tool_call handler runs the tool. Commit its result once and emit a follow-up llm effect.
4. The second llm handler runs with the tool result. Commit the final assistant answer.
5. With no remaining effects or in-flight work, settle the session as done.

Between any two steps, every worker could disappear. A new worker can continue from the last committed state and the remaining queued effects.

A crash becomes a replay

A worker claims an effect, receives a claim_token, and gets a bounded lease. Suppose it begins servicing the effect and then dies. The transaction that would have advanced state never commits. Nothing durable changed.

When the lease expires, the effect becomes visible again. Another worker claims it with a new token and runs from the last committed state.

What if the original worker was only slow? It may return and try to settle the effect after another worker has reclaimed it. Its token is stale, so the settlement becomes a no-op. Idempotent result mutations provide another line of defense against duplicate state changes.

This is how a crash becomes a replay.

This brings us back to the nine-hour stall.

The durable layer had not lost any work. State and effects remained in Postgres. The polling loop had treated a transient database failure as a reason to stop draining, then stayed alive long enough that no supervisor replaced it.

The fix followed from the execution model. A transient database error should cause backoff and retry. If failures continue beyond a bounded schedule, the worker should throw and exit so its supervisor can restart it. Either route preserves the same invariant: queued work remains durable. Recovery requires restarting a worker, not reconstructing agent sessions.

Where the analogy stops

Conol is not an algebraic-effects system.

We do not have first-class delimited continuations. When a worker resumes, it does not restore a captured stack. It reconstructs context from durable state and interprets another effect.

We do not have an effect algebra enforced by the system, handler stacking with lexical scope, or effect-row polymorphism. Our handlers are entries in a registry. Effects such as llm and tool_call are coarse jobs that can run for minutes across processes, closer to actor messages or saga steps than language-level operations such as get and raise.

The closest analogue is human input. The AskUser tool leaves its tool call pending and allows the queue to drain. Later, a reply effect supplies the answer. Its mutation completes the pending result and emits another llm effect. That behaves like suspension and resumption, but the mechanism is persistent data rather than a captured continuation.

In ordinary systems terms, Conol is a reducer that returns commands, backed by a transactional outbox and leased queue. Its design is inspired by effect handlers.

Design rules

The runtime should become less interesting

The goal is not cleverness. It is boredom.

A good agent runtime should be the least interesting part of the system. The model does the interesting work. The runtime makes sure that when a process dies, hangs, or a database restarts, recovery means starting another worker rather than reconstructing partially completed turns.

We got there by borrowing one idea from programming-language theory: separate what a computation requests from how that request is interpreted. Then we implemented the separation with durable primitives we already trusted: a transaction, a queue, a lease, and idempotent state transitions.

No captured continuations. No effect type system. Just data that outlives the process that produced it.

A stack belongs to a process. An agent turn should not.

References

1. Moggi, E. (1991). Notions of Computation and Monads. Information and Computation, 93(1), 55–92.
2. Plotkin, G., & Power, J. (2003). Algebraic Operations and Generic Effects. Applied Categorical Structures, 11, 69–94.
3. Plotkin, G., & Pretnar, M. (2009). Handlers of Algebraic Effects. PDF
4. Pretnar, M. (2015). An Introduction to Algebraic Effects and Handlers. PDF
5. OCaml. Effect handlers. Manual
6. Sivaramakrishnan, K. C., et al. (2021). Retrofitting Effect Handlers onto OCaml. arXiv
7. Leijen, D. The Koka Language Book. Book
8. OCaml Multicore. Eio: Effects-based direct-style I/O for OCaml. GitHub