kalayciburak commented on PR #12678: URL: https://github.com/apache/seatunnel/pull/12678#issuecomment-6937943365
yes, at `71ac656652f0` pruning runs after each successful completion once the deque is over the limit. If `deleteCheckpoint` reports an `Exception`, those ids go back to the front in order and are retried on the next eligible completion. Repeated failures can grow the queue. This only holds for the current coordinator and reported failures; some storage plugins swallow individual file-delete errors. I meant your [Oct 28 comment](https://github.com/apache/seatunnel/pull/12678#issuecomment-6459609745): "Keeping restore seeding out of this PR and documenting the limitation plus the follow-up in the PR description works for the scope here." The current description says a new coordinator starts with an empty deque and links #12094. It doesn't spell out the full impact: old files are not selected by the new coordinator; after one restart the total can reach roughly 2x the limit, and repeated restarts can take it higher. #12094 already describes restoring IDs from storage after coordinator recreation. The storage-backed retry test and exact-head Build are still pending. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
