purushah commented on PR #1138: URL: https://github.com/apache/flink-agents/pull/1138#issuecomment-5736995548
Thanks for picking this up. Before we get into line-by-line review, could we open a discussion for the runtime model? This PR settles a few things #909 left open, and they affect most of the operator changes. The three I'd like agreement on: 1. Where in-flight call state lives across a checkpoint. Right now it's only on the heap and rebuilt when the parent replays, but child tasks are checkpointed before the parent is re-enqueued. On restore the child can run first, find no status, and the job loops. 2. Whether the parent should wait on a `ContinuationActionExecutor` thread. Children use the same pool, so enough concurrent calls on one subtask can starve it. 3. What "done" means for a child with more than one action. Replay only re-sends the child's output events, so a chained child comes back with partial output after failover. I have more detailed notes and am happy to share them once the direction is set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
