SEZ9 commented on PR #12096: URL: https://github.com/apache/seatunnel/pull/12096#issuecomment-6962288625
Thanks for the status updates. The successful fork Build 41162813818 on `aa30f22ef2bd316cb707d89ee233e183a152a0d5` is good to have, but as the thread already notes, it does not resolve the earlier review points. No revised code has been posted since, so all of them stay open. What I'd like to see in the next revision: - **Restore safety (F1):** constructor-time retention cleanup must never delete the checkpoint or savepoint the job is being restored from. Please add a recovery test that proves the selected restore point survives pruning. - **Savepoints (F2, F4):** user-triggered savepoints must be excluded from automatic retention when the durable pipeline states are enumerated after a restart or failover, with a test covering that. - **Partial cleanup failure (F3):** a partially deleted batch must not wedge retention cleanup, and the retry path must not build an unbounded backlog when deletion keeps failing. Please cover this with a test too. - **Recovery cost (F5):** bound the materialization during the recovery scan, so the coordinator constructor does not synchronously deserialize the full state payload of every retained checkpoint. - **Exception handling (F7):** a `RuntimeException` from storage during retention restore or cleanup should not escape the `CheckpointCoordinator` constructor and abort coordinator recreation during active-master failover. - **Docs and compatibility (F6, F8):** document the changed `checkpoint.storage.max-retained` behavior. That means exactly max-retained kept (previously up to 2*max-1) and one `deleteCheckpoint` storage call per completed checkpoint. The thread does not yet say how you plan to handle F7, F6 and F8. Could you confirm whether they will be in the same revision? I'll review again once the revised code is pushed. <!-- streview-comment:2413 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
