SEZ9 commented on PR #12096: URL: https://github.com/apache/seatunnel/pull/12096#issuecomment-6870936853
Thanks for the follow-up on unchanged head `aa30f22ef2bd316cb707d89ee233e183a152a0d5`. Agreed: successful fork Build 41162813818 does not clear the source-level findings, which remain open on this head. To restate what still needs to change in the constructor-time retention path if this PR remains the selected implementation: - **Restore target (HIGH)**: retention cleanup must never delete the checkpoint/savepoint the job is currently being restored from; please exclude it explicitly from the eligible-for-deletion set. - **Savepoint classification**: retention restore enumerates all durable pipeline states, so user-created savepoints become eligible for deletion after restart/failover. Savepoints need to be distinguished from checkpoints and excluded from pruning. - **Partial batch failure / retry**: cleanup retry assumes `deleteCheckpoint` is idempotent; a partially-deleted batch should not wedge retention cleanup, and the pending-deletion queue must not grow without bound when a deletion keeps failing. - **Exception handling**: only `CheckpointStorageException` is caught; a `RuntimeException` from storage can escape the constructor and abort coordinator recreation during active-master failover. Please make this path best-effort. - **Recovery-time cost**: deserializing the full state payload of every retained checkpoint synchronously in the constructor slows active-master recovery; a metadata-only enumeration would address this. - **Docs**: `checkpoint.storage.max-retained` semantics changed (exactly max-retained kept vs. previously up to 2*max-1, plus one `deleteCheckpoint` call per completed checkpoint); the user-facing documentation needs updating to match. On the overlap question: as noted, reconciliation of the overlapping storage-retention PRs is a maintainer decision and remains separate from this review. Once that direction is confirmed, a focused revision addressing the items above, plus unit tests covering restore-target exclusion, savepoint exclusion, and partial-deletion-failure retry, is what would unblock this PR. This is a discussion reply, not a new full source review or an approval. No local tests were run. <!-- streview-comment:2342 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
