Thanks Elijah, this is the right question to push on. Let me answer it head
on, because the answer is "yes there is an easier way, and I think we
should ship it as the default."
You are right that a durable state can ride inside the existing state model
rather than a parallel structure. PR #786 already proves this: custom
persisters fall back to an in-state journal, and that path is correct. No
new persister methods, no dedicated tables, onboarding is zero. For a
single-worker workflow it is the simplest thing that works, and I do not
want to force anyone onto a heavier model to get suspend/resume.
So I want to reframe the design as two tiers rather than one monolith:
- *Default (light):* journal and suspension records live in the existing
state keyspace, exactly as you suggested. Any persister gets durable
execution for free. This is the onboarding path.
- *Opt-in (heavy):* a persister that implements the dedicated methods (
save_suspension, mark_suspension_resolved, etc.) for workloads that
outgrow the light path.
The reason the opt-in tier earns its keep is not aesthetics, it is four
concrete costs the in-state path hits at scale:
1. *State blob growth.* Journal entries and suspension payloads inside
state mean every save reserializes the whole blob. A workflow with N
memoized sub-steps rewrites O(N) state on every step. Fine for small flows,
painful for long ones.
2. *Resume-once under concurrency.* Two workers resuming the same
suspension need an atomic claim. The SQL persisters expose exactly that
primitive: mark_suspension_resolved is a conditional UPDATE ... WHERE
resolved = 0 returning rowcount > 0, so under concurrency exactly one
caller wins. In-state cannot express that primitive at all. To be precise
about the current state of the branch: resume() / aresume() still gate
on a non-atomic record.resolved read taken before the run and discard
the bool, so the race is not yet closed end to end. Wiring resumption to
skip the run unless mark_suspension_resolved returns True is part of
this work, and it is only expressible on the dedicated tier (the in-memory
persister is a plain loop, not atomic).
3. *Queryability.* "List all pending suspensions awaiting approval" is
one SQL query against a table. In-state it is a full scan of every app's
serialized state. Not viable for an ops dashboard or the Burr UI.
4. *Schema coupling.* Mixing runtime metadata into the user state
keyspace invites key collisions and muddies "what is my state" versus "what
is runtime bookkeeping."
*The real case driving this.* A loyalty CRM I am running in production. A
campaign action generates a personalized offer per customer via an LLM,
processed as a sequential batch of ~50. Two things break today:
- Item 23 needs human approval before the offer goes out. Items 1
through 22 have already paid for their LLM calls. With halt_before/
halt_after I cannot pause mid batch, so I either fork state manually or
re-run the batch and pay for 1 through 22 again.
- On any crash or retry mid batch, every offer regenerates. The
offer-generation LLM is the dominant cost in the workflow, so a single
retry doubles the spend for that run.
With __context.durable("offer", generate_offer, customer) the replay reads
the recorded offers and re-fires nothing. This is test-backed in the PR:
the integration suite asserts a durable side effect runs exactly once
across a suspend/resume cycle (the recorded value is replayed, the function
is not re-invoked). I also ran the HITL example end to end against a local
model to confirm the same behavior outside the test harness.
Now, here is where the in-state default would actually hurt this case,
which is why I want the opt-in tier available: the recorded offer payloads
sit in the state blob and get reserialized on every subsequent step, and
the approval step is exactly where a second worker could pick up the same
suspension. Costs 1 and 2 are not hypothetical for this workload.
*On the API surface.* I like your Azure-style yield ctx.durable(...) with
match. It reads cleanly. The current PR uses the callable form ctx.durable(key,
fn, ...); I am open to the generator form if folks prefer it, the journal
semantics are the same underneath. And halt_when(durable=[...]) as thin
sugar over conditional edges is a nice touch, that also answers Stefan's
preference to keep halting expressed as state plus edges rather than a new
transition primitive.
*On Stefan's hierarchy point:* the journal key is (partition_key, app_id,
sequence_id, step_key) today, with entries ordered by call_index. It does
not yet fold in a parent app id, so nested apps do not inherit a parent
journal. I will make the key compose with the parent app id so nested apps
inherit the journal correctly, which is the hierarchical reuse you flagged.
Proposed path: keep suspend/resume and the journal in #786, make the
in-state path the documented default, mark the dedicated persister methods
as the opt-in scale tier, and address hierarchy in the same PR. If that
shape works for both of you I will split out anything that does not need to
land together.
No deadline. Will send a [PROPOSAL] once we converge.
PS: sorry for the blogpost in dev list, but is a big theme. :)
André
Em dom., 31 de mai. de 2026 às 22:58, Elijah ben Izzy <
[email protected]> escreveu:
> This is very cool. So I thought through there's definitely a need here to
> pause mid-action.
>
> Some API ideas:
> 1. Put durable_keys as part of the declaration to match the type-safe,
> declarative nature of the rest of it
> 2. Use something clever (i.e. match statements) to make it easier to match
> the mental model
>
> Azure durable functions has an interesting approach:
>
> def my_action(state, ctx):
> user = yield ctx.durable("fetch", db.get_user, state["uid"])
> match user.tier:
> case "gold":
> result = yield ctx.durable("gold_flow", gold_flow, user)
> case _:
> result = yield ctx.durable("std_flow", std_flow, user)
> return state.update(result=result)
>
> Worth exploring. I'm also wondering if there's a simple way to add an
> optional field to the data model rather than a whole new "supports
> durable"? I.E. would there be an easier way to onboard? Could we model it
> as some sort of sub-state that we could store and pass in? If we had
> durable things as a special key-space in the state could we just use the
> same ones? Trade-offs not sure the right one.
>
> The problem is that if we don't have suspend() we decouple it but need a
> mechanism to halt execution. Might be a good place for syntactic sugar --
> halt_during? halt_when(durable=["key"])?
>
> With iterator-based APIs it's actually up to the user to handle the halting
> -- this completely bypasses this issue (we just send out a durable-set
> event mid-stride as we would an iterator or an iterator of streams).
>
>
>
> On Sun, May 31, 2026 at 11:45 AM Stefan Krawczyk <
> [email protected]>
> wrote:
>
> > If halt_when has clear semantics if what happens next then that sounds
> > good. But I haven't found a clear way to make that obvious. So I think
> > making users set up state and conditional edges should be the cleaner
> model
> > (we should also show that running a graph with halt* is fine in examples
> > and drop that warning).
> >
> > Durable key seems fine I think. I think the hierarchical nature should be
> > utilized that we've set up, so more capabilities to enable that way of
> > pausing and resuming to be more useful makes sense to me -- IIUC.
> >
> > On Thu, May 28, 2026, 6:14 AM André Ahlert <[email protected]> wrote:
> >
> > > Hi all,
> > >
> > > PR #786 <https://github.com/apache/burr/pull/786> [1] proposes two
> > > additions to handle long-running agent workloads. Moving the design
> > > question from DM with Stefan onto the list before we iterate further.
> > >
> > > Gap today: halt_before / halt_after only stop between actions. Two
> cases
> > > keep showing up:
> > >
> > > 1. HITL inside one action (e.g. sequential batch where item 23 of 50
> > > needs approval, items 1-22 already paid for LLM calls).
> > > 2. Crash recovery mid-action without re-firing paid sub-steps.
> > >
> > > Langgraph covers both with interrupt() + checkpointer. Concrete case I
> am
> > > hitting: a loyalty CRM where every retry re-fires the offer-generation
> > LLM.
> > >
> > > Proposal is to split #786 into two orthogonal pieces, ship
> independently:
> > >
> > > * A. *halt_when(predicate): state-based HITL primitive. Action sets a
> > state
> > > value, runtime halts when predicate matches, conditional edges route on
> > > resume. No new resume semantics. (Stefan's suggested shape on the PR.)
> > >
> > > * B. *__context.durable(key, fn): sub-step memoization journal. Result
> > > keyed by (app_id, sequence, key). On replay, returns recorded value
> > without
> > > re-firing fn. Defensible on crash recovery alone.
> > >
> > > Worked example (before/after code, mid-action case discussion):
> > > https://gist.github.com/
> > >
> > > Open questions:
> > >
> > > 1. halt_when(predicate) as first-class transition primitive, or keep
> > as
> > > user-side conditional edge expression?
> > > 2. __context.durable(key, fn) the right surface, or decorator?
> > > 3. Anyone hitting the mid-action sequential case in production?
> > >
> > > Not proposing the suspend()-style mid-action pause in this thread. Want
> > to
> > > ship A and B, collect signal, revisit on a separate [DISCUSS] if demand
> > > shows up.
> > >
> > > No deadline. Will follow up with [PROPOSAL] per piece once we converge.
> > >
> > > [1] https://github.com/apache/burr/pull/786
> > >
> > > Thanks, André
> > >
> > >
> >
>
