DanielLeens commented on PR #13239: URL: https://github.com/apache/seatunnel/pull/13239#issuecomment-7007011832
Thanks for expanding both examples. +1 to @nzw921rx's ownership/completion points and to publishing the enumerator snapshot on its assignment path. The revised in-flight name and explicit split-return behavior are clearer. One inconsistency remains in the proposal: the text says activeSplitId identifies the split that produced the position and that the pair changes together, but row 3 changes it to b.csv while retaining a.csv's position. Keeping the last emitted position is useful; please keep its owning split ID as a.csv too until the first record from b.csv is emitted. Alternatively clear both the position and its owner on the switch. Do not expose a mismatched owner/position pair. This is clarification of the proposal in this thread, not a new implementation blocker on the unchanged pushed API. The completion/idle publication should publish the latest state without inventing a new lastPositionChangeAt, and the reassignment example should remain the basis of the future engine stale-attempt test. The planned contract changes are not yet in the head, so I will keep the existing API/design review open until the coordinated #13239/#13240/#11512 commits are pushed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
