DanielLeens commented on issue #13224: URL: https://github.com/apache/seatunnel/issues/13224#issuecomment-6795352958
Thanks for recording these as explicit follow-ups rather than widening #12440. The linked exactly-once PR is still open and its Build is in progress, so this Issue should remain a follow-up lane and must not rework the in-flight implementation. The two follow-ups are both necessary. #12440 currently carries `List<WriteStatus>` in its checkpoint commit information, and its E2E verifies visibility after a successful job but does not exercise a restart between checkpoint completion and aggregated commit. Please begin this work only on the merged base, in two focused changes: 1. Replace the Hudi-client `WriteStatus` checkpoint payload with a connector-owned, versioned state representation that contains exactly the information required by the Hudi commit call. Add serialization and restore tests that prove a completed checkpoint can still be committed after recovery; do not silently drop any commit information during a state-format transition. 2. Add a fault-injection E2E that stops the job after the checkpoint is completed and handed to the aggregated committer but before the Hudi instant is committed. Restart from that checkpoint and assert the final table contents and committed instant set: no missing rows, no duplicate rows, no publication of the uncompleted attempt, and the expected reclaim/rollback behavior for orphaned instants. Please keep the default `AT_LEAST_ONCE` path outside this follow-up unless a regression is demonstrated, and report the exact engine coverage and failure hook used by the E2E in the PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
