fishfishfishfishaa commented on PR #8671: URL: https://github.com/apache/paimon/pull/8671#issuecomment-6812616106
Thanks for the review. I've updated the recovery logic, added more ITs, and simplified the writer state. For partially finished recovery, the coordinator keeps terminal coverage for writers that remain finished. If a previously covered writer becomes active again after restore, its old coverage is cleared and it participates in commits and EndInput again. MAX finalization also waits for a subsequent completed checkpoint, rather than relying only on historical coverage. Restoring terminal coverage with a different writer parallelism is explicitly rejected. For fully finished recovery, the last writer only finishes after the data commits, relevant tags, MAX finalization, and listeners succeed. If Flink restores the operator without coordinator state, it enters `DORMANT`, allowing later checkpoints without new table commits. A fresh start is distinguished by the absence of a restored checkpoint ID. This relies on the last writer’s completion guarantee; empty coordinator state itself is not treated as proof of commit success. Regarding commit semantics, I agree that coordinator commit is not exactly equivalent to operator commit. I still keep commit progress and recovery metadata in coordinator state, but pending committables stay in writer state and are replayed on recovery, since the coordinator snapshots before the writers capture their state for that checkpoint. The reason for the two-step final commit is to keep the final data on the normal checkpoint recovery path. For checkpoint K, the data is first committed under K, followed by an empty MAX commit for global EndInput finalization. If we instead put the final data directly into MAX and fail after that snapshot is published, recovery still replays the data under K. We would need extra handling to establish that MAX already contains that data, rather than submitting it again under K. Keeping the data committed under K lets recovery use the existing checkpoint-based filtering, then retry MAX finalization separately. It also gives savepoint tags a snapshot with the actual checkpoint ID. A single-snapshot approach may be possible, but I chose this approach to avoid adding a separate recovery protocol for the final data. The trade-off is that data can become visible before global finalization, and savepoint tags point to the checkpoint snapshot rather than MAX. The original operator path can also expose data before finalization during failover, although that does not make the normal execution semantics equivalent. I've documented these differences. Do you think this trade-off is reasonable? I also consolidated the per-writer state into `WriterState`, removed redundant state, and kept the additional watermark behavior and classic committer changes out of this PR. Added 17 IT cases, counting parameterized invocations, covering partially and fully finished recovery, global and region failover strategies, failures around MAX publication, draining savepoint/tag recovery, and partition completion options. Both local Flink 1 and Flink 2 test runs passed 144 tests each, including existing regression and unit tests. Remote CI is pending, and process-level JM/HA failures are not covered. Please take another look. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
