Skip to main content

Durable Execution

Long-running agents outlive the processes that start them. WorkflowEnvelope is the execution boundary that makes a run survivable: it checkpoints progress so a crash, a redeploy, or a pause for human input resumes where it left off instead of starting over.

You don't construct an envelope — the runtime wraps your workflow in one. What you control is where the checkpoints fall.

What creates a checkpoint​

Every tool call is a durable step:

from waxell_runtime import agent, workflow, tool


@agent(name="report_builder", description="Builds a quarterly report")
class ReportBuilder:

@tool
async def fetch_data(self, ctx, quarter: str) -> dict:
return await ctx.domain("finance", "quarterly", quarter=quarter)

@tool
async def summarize(self, ctx, data: dict) -> str:
return await ctx.llm.generate(prompt=f"Summarize: {data}")

@workflow("build")
async def build(self, ctx, quarter: str):
# Each ctx.tool(...) is a checkpoint boundary.
data = await ctx.tool("fetch_data", quarter=quarter)

# If the process dies here, resuming replays the line above from the
# saved result rather than re-fetching.
summary = await ctx.tool("summarize", data=data)

return {"quarter": quarter, "summary": summary}

Two things to note, because they're the usual mistakes:

  • Everything is async. ctx.tool, ctx.domain, and ctx.llm.generate are all awaited.
  • Checkpointing is automatic. There's no durable=True flag and no separate "durable workflow" type — a workflow's tool calls are its checkpoints.

Work done between tool calls is not checkpointed. If a computation is expensive enough that you'd hate to repeat it, put it in a @tool so its result is saved.

How resume works​

  1. Before a step runs, the runtime records that it's starting.
  2. After it completes, the result is checkpointed.
  3. On resume, any step with a saved result is replayed from that result — the function does not run again.
  4. Execution continues from the first step without one.

This is why tools should be the unit of work: the checkpoint granularity is the tool boundary.

Run states​

A run's lifecycle, from RunState:

StateMeaningTerminal
PENDINGAccepted, not yet started
RUNNINGExecuting
PAUSEDWaiting on something external
RESUMINGPicking back up from a checkpoint
COMPLETEDFinished successfully✅
FAILEDAn error inside the agent✅
BLOCKEDStopped by governance — a policy or budget✅
INTERRUPTEDThe process died (SIGTERM, OOM, hard kill) before finishing✅
Poll for all four terminal states

BLOCKED and INTERRUPTED are terminal. Code that waits for only COMPLETED or FAILED will hang forever on a policy-blocked or process-killed run.

INTERRUPTED is deliberately distinct from FAILED: an agent that errored is a different problem from a process that was killed under it, and collapsing them hides infrastructure issues.

Why a run is paused​

PAUSED isn't one thing — PausedReason tells you what it's waiting on:

ReasonWaiting for
awaiting_childA spawned sub-agent
awaiting_signalAn inbound signal
awaiting_userA human answer (ctx.ask_user)
awaiting_timerA scheduled wake-up
awaiting_approvalA governance approval
suspended_orphanIts parent went away

A paused run holds no compute. It resumes when the thing it's waiting on arrives.

Where state is stored​

The storage backend is chosen by which RuntimeBackend is configured, not by a separate setting:

  • Local development — InMemoryBackend, automatic, nothing to configure. State lives in the process, so it does not survive a restart.
  • Production — DjangoRuntimeBackend, wired automatically when waxell_infra is in INSTALLED_APPS. Checkpoints persist to the database and survive restarts and redeploys.

See Backends for how that wiring happens and how to supply your own.

Next Steps​