Can we start some gists? Or draft PRs? That would help me if we talked UX and what it would mean. E.g. let's take that loyalty CRM case and look at the API one would use to implement it.
On Mon, Jun 1, 2026, 12:41 PM André Ahlert <[email protected]> wrote: > Thanks Elijah, this is the right question to push on. Let me answer it head > on, because the answer is "yes there is an easier way, and I think we > should ship it as the default." > > You are right that a durable state can ride inside the existing state model > rather than a parallel structure. PR #786 already proves this: custom > persisters fall back to an in-state journal, and that path is correct. No > new persister methods, no dedicated tables, onboarding is zero. For a > single-worker workflow it is the simplest thing that works, and I do not > want to force anyone onto a heavier model to get suspend/resume. > > So I want to reframe the design as two tiers rather than one monolith: > > - *Default (light):* journal and suspension records live in the existing > state keyspace, exactly as you suggested. Any persister gets durable > execution for free. This is the onboarding path. > - *Opt-in (heavy):* a persister that implements the dedicated methods ( > save_suspension, mark_suspension_resolved, etc.) for workloads that > outgrow the light path. > > The reason the opt-in tier earns its keep is not aesthetics, it is four > concrete costs the in-state path hits at scale: > > 1. *State blob growth.* Journal entries and suspension payloads inside > state mean every save reserializes the whole blob. A workflow with N > memoized sub-steps rewrites O(N) state on every step. Fine for small > flows, > painful for long ones. > 2. *Resume-once under concurrency.* Two workers resuming the same > suspension need an atomic claim. The SQL persisters expose exactly that > primitive: mark_suspension_resolved is a conditional UPDATE ... WHERE > resolved = 0 returning rowcount > 0, so under concurrency exactly one > caller wins. In-state cannot express that primitive at all. To be > precise > about the current state of the branch: resume() / aresume() still gate > on a non-atomic record.resolved read taken before the run and discard > the bool, so the race is not yet closed end to end. Wiring resumption to > skip the run unless mark_suspension_resolved returns True is part of > this work, and it is only expressible on the dedicated tier (the > in-memory > persister is a plain loop, not atomic). > 3. *Queryability.* "List all pending suspensions awaiting approval" is > one SQL query against a table. In-state it is a full scan of every app's > serialized state. Not viable for an ops dashboard or the Burr UI. > 4. *Schema coupling.* Mixing runtime metadata into the user state > keyspace invites key collisions and muddies "what is my state" versus > "what > is runtime bookkeeping." > > *The real case driving this.* A loyalty CRM I am running in production. A > campaign action generates a personalized offer per customer via an LLM, > processed as a sequential batch of ~50. Two things break today: > > - Item 23 needs human approval before the offer goes out. Items 1 > through 22 have already paid for their LLM calls. With halt_before/ > halt_after I cannot pause mid batch, so I either fork state manually or > re-run the batch and pay for 1 through 22 again. > - On any crash or retry mid batch, every offer regenerates. The > offer-generation LLM is the dominant cost in the workflow, so a single > retry doubles the spend for that run. > > With __context.durable("offer", generate_offer, customer) the replay reads > the recorded offers and re-fires nothing. This is test-backed in the PR: > the integration suite asserts a durable side effect runs exactly once > across a suspend/resume cycle (the recorded value is replayed, the function > is not re-invoked). I also ran the HITL example end to end against a local > model to confirm the same behavior outside the test harness. > > Now, here is where the in-state default would actually hurt this case, > which is why I want the opt-in tier available: the recorded offer payloads > sit in the state blob and get reserialized on every subsequent step, and > the approval step is exactly where a second worker could pick up the same > suspension. Costs 1 and 2 are not hypothetical for this workload. > > *On the API surface.* I like your Azure-style yield ctx.durable(...) with > match. It reads cleanly. The current PR uses the callable form > ctx.durable(key, > fn, ...); I am open to the generator form if folks prefer it, the journal > semantics are the same underneath. And halt_when(durable=[...]) as thin > sugar over conditional edges is a nice touch, that also answers Stefan's > preference to keep halting expressed as state plus edges rather than a new > transition primitive. > > *On Stefan's hierarchy point:* the journal key is (partition_key, app_id, > sequence_id, step_key) today, with entries ordered by call_index. It does > not yet fold in a parent app id, so nested apps do not inherit a parent > journal. I will make the key compose with the parent app id so nested apps > inherit the journal correctly, which is the hierarchical reuse you flagged. > > Proposed path: keep suspend/resume and the journal in #786, make the > in-state path the documented default, mark the dedicated persister methods > as the opt-in scale tier, and address hierarchy in the same PR. If that > shape works for both of you I will split out anything that does not need to > land together. > > No deadline. Will send a [PROPOSAL] once we converge. > > PS: sorry for the blogpost in dev list, but is a big theme. :) > > André > > Em dom., 31 de mai. de 2026 às 22:58, Elijah ben Izzy < > [email protected]> escreveu: > > > This is very cool. So I thought through there's definitely a need here to > > pause mid-action. > > > > Some API ideas: > > 1. Put durable_keys as part of the declaration to match the type-safe, > > declarative nature of the rest of it > > 2. Use something clever (i.e. match statements) to make it easier to > match > > the mental model > > > > Azure durable functions has an interesting approach: > > > > def my_action(state, ctx): > > user = yield ctx.durable("fetch", db.get_user, state["uid"]) > > match user.tier: > > case "gold": > > result = yield ctx.durable("gold_flow", gold_flow, user) > > case _: > > result = yield ctx.durable("std_flow", std_flow, user) > > return state.update(result=result) > > > > Worth exploring. I'm also wondering if there's a simple way to add an > > optional field to the data model rather than a whole new "supports > > durable"? I.E. would there be an easier way to onboard? Could we model it > > as some sort of sub-state that we could store and pass in? If we had > > durable things as a special key-space in the state could we just use the > > same ones? Trade-offs not sure the right one. > > > > The problem is that if we don't have suspend() we decouple it but need a > > mechanism to halt execution. Might be a good place for syntactic sugar -- > > halt_during? halt_when(durable=["key"])? > > > > With iterator-based APIs it's actually up to the user to handle the > halting > > -- this completely bypasses this issue (we just send out a durable-set > > event mid-stride as we would an iterator or an iterator of streams). > > > > > > > > On Sun, May 31, 2026 at 11:45 AM Stefan Krawczyk < > > [email protected]> > > wrote: > > > > > If halt_when has clear semantics if what happens next then that sounds > > > good. But I haven't found a clear way to make that obvious. So I think > > > making users set up state and conditional edges should be the cleaner > > model > > > (we should also show that running a graph with halt* is fine in > examples > > > and drop that warning). > > > > > > Durable key seems fine I think. I think the hierarchical nature should > be > > > utilized that we've set up, so more capabilities to enable that way of > > > pausing and resuming to be more useful makes sense to me -- IIUC. > > > > > > On Thu, May 28, 2026, 6:14 AM André Ahlert <[email protected]> wrote: > > > > > > > Hi all, > > > > > > > > PR #786 <https://github.com/apache/burr/pull/786> [1] proposes two > > > > additions to handle long-running agent workloads. Moving the design > > > > question from DM with Stefan onto the list before we iterate further. > > > > > > > > Gap today: halt_before / halt_after only stop between actions. Two > > cases > > > > keep showing up: > > > > > > > > 1. HITL inside one action (e.g. sequential batch where item 23 of > 50 > > > > needs approval, items 1-22 already paid for LLM calls). > > > > 2. Crash recovery mid-action without re-firing paid sub-steps. > > > > > > > > Langgraph covers both with interrupt() + checkpointer. Concrete case > I > > am > > > > hitting: a loyalty CRM where every retry re-fires the > offer-generation > > > LLM. > > > > > > > > Proposal is to split #786 into two orthogonal pieces, ship > > independently: > > > > > > > > * A. *halt_when(predicate): state-based HITL primitive. Action sets a > > > state > > > > value, runtime halts when predicate matches, conditional edges route > on > > > > resume. No new resume semantics. (Stefan's suggested shape on the > PR.) > > > > > > > > * B. *__context.durable(key, fn): sub-step memoization journal. > Result > > > > keyed by (app_id, sequence, key). On replay, returns recorded value > > > without > > > > re-firing fn. Defensible on crash recovery alone. > > > > > > > > Worked example (before/after code, mid-action case discussion): > > > > https://gist.github.com/ > > > > > > > > Open questions: > > > > > > > > 1. halt_when(predicate) as first-class transition primitive, or > keep > > > as > > > > user-side conditional edge expression? > > > > 2. __context.durable(key, fn) the right surface, or decorator? > > > > 3. Anyone hitting the mid-action sequential case in production? > > > > > > > > Not proposing the suspend()-style mid-action pause in this thread. > Want > > > to > > > > ship A and B, collect signal, revisit on a separate [DISCUSS] if > demand > > > > shows up. > > > > > > > > No deadline. Will follow up with [PROPOSAL] per piece once we > converge. > > > > > > > > [1] https://github.com/apache/burr/pull/786 > > > > > > > > Thanks, André > > > > > > > > > > > > > >
