tagadearpit opened a new pull request, #12681: URL: https://github.com/apache/seatunnel/pull/12681
Fixes #12677 ### Purpose of this pull request `checkpoint.storage.max-retained` is documented as the maximum number of retained checkpoints. `CheckpointCoordinator.completePendingCheckpoint` only triggered cleanup when the completed checkpoint ID queue size was an exact multiple of `max-retained` and greater than 1, causing storage to hold up to `2 * max-retained - 1` checkpoints between prunes. This fix trims the oldest checkpoint IDs continuously as soon as a completion exceeds `max-retained`. The configuration name, default (20), and documentation remain unchanged. ### Does this PR introduce _any_ user-facing change? Yes, behavior change only. With `max-retained: 3`, storage previously retained up to 5 checkpoints before a batch delete. It now retains at most 3 checkpoints upon completion of each checkpoint. No configuration options or public APIs were renamed or removed. ### How was this patch tested? - Added `CheckpointCoordinatorTest#testRetentionPrunesAsSoonAsCompletedCheckpointsExceedMaxRetained` asserting that `completedCheckpointIds.size() <= maxRetained` after completing checkpoints 1 to 5 with `max-retained = 3`, and verifying that checkpoints 1 and 2 are pruned. - Updated `CheckpointStorageRetentionPruneTest` on-disk bound assertion from `<= 2 * maxRetained` to `<= maxRetained + 1`. ### Check list * [x] No new Jar binary package added. * [x] No documentation changes required (existing docs already specify max-retained as the maximum). * [x] No incompatible changes introduced. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
