joeyutong commented on issue #1016: URL: https://github.com/apache/flink-agents/issues/1016#issuecomment-5679858238
I agree with keeping `durable_execute_async` lazy. Submission on await is already part of its public contract, so I think it is reasonable for `ctx.durable_gather(...)` to rely on that. One implementation detail we should account for is recovery matching. Without a reconciler, `durable_execute_async` currently matches persisted results before the returned `AsyncExecutionResult` is awaited. This works with the existing `await ctx.durable_execute_async(...)` pattern: recovery proceeds sequentially, and the cursor is advanced before the next `durable_execute_async` is evaluated. With the proposed `durable_gather` shape, however, multiple `AsyncExecutionResult` objects are created before `durable_gather` starts planning the batch. Recovery matching during their creation could advance the cursor or clear subsequent results before the whole batch is prepared. I would therefore defer recovery matching until await as well: standalone `await ctx.durable_execute_async(...)` would use the existing serial recovery path, while `ctx.durable_gather(...)` would coordinate recovery matching and reserve the whole batch’s pending slots before execution. This would preserve the reserve-before-execute mechanism. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
