SEZ9 commented on PR #12096:
URL: https://github.com/apache/seatunnel/pull/12096#issuecomment-6870936853

   Thanks for the follow-up on unchanged head 
`aa30f22ef2bd316cb707d89ee233e183a152a0d5`. Agreed: successful fork Build 
41162813818 does not clear the source-level findings, which remain open on this 
head.
   
   To restate what still needs to change in the constructor-time retention path 
if this PR remains the selected implementation:
   
   - **Restore target (HIGH)**: retention cleanup must never delete the 
checkpoint/savepoint the job is currently being restored from; please exclude 
it explicitly from the eligible-for-deletion set.
   - **Savepoint classification**: retention restore enumerates all durable 
pipeline states, so user-created savepoints become eligible for deletion after 
restart/failover. Savepoints need to be distinguished from checkpoints and 
excluded from pruning.
   - **Partial batch failure / retry**: cleanup retry assumes 
`deleteCheckpoint` is idempotent; a partially-deleted batch should not wedge 
retention cleanup, and the pending-deletion queue must not grow without bound 
when a deletion keeps failing.
   - **Exception handling**: only `CheckpointStorageException` is caught; a 
`RuntimeException` from storage can escape the constructor and abort 
coordinator recreation during active-master failover. Please make this path 
best-effort.
   - **Recovery-time cost**: deserializing the full state payload of every 
retained checkpoint synchronously in the constructor slows active-master 
recovery; a metadata-only enumeration would address this.
   - **Docs**: `checkpoint.storage.max-retained` semantics changed (exactly 
max-retained kept vs. previously up to 2*max-1, plus one `deleteCheckpoint` 
call per completed checkpoint); the user-facing documentation needs updating to 
match.
   
   On the overlap question: as noted, reconciliation of the overlapping 
storage-retention PRs is a maintainer decision and remains separate from this 
review. Once that direction is confirmed, a focused revision addressing the 
items above, plus unit tests covering restore-target exclusion, savepoint 
exclusion, and partial-deletion-failure retry, is what would unblock this PR.
   
   This is a discussion reply, not a new full source review or an approval. No 
local tests were run.
   
   <!-- streview-comment:2342 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to