SEZ9 commented on issue #13017: URL: https://github.com/apache/seatunnel/issues/13017#issuecomment-6461185025
Thanks for the well-reasoned proposal and the data from #12022. Since #12387 rebuilds the counter via setCount(latestCompletedCheckpointId + 1), excluding this map from persistence looks safe. Please go ahead with the PR and include a test covering full-cluster restart with (a) a pipeline that has completed checkpoints and (b) a pipeline with none, asserting the counter resumes correctly in both cases. One edge case to confirm in the PR: a pipeline whose checkpoints were triggered but none completed (e.g. checkpoint 1..N pending/aborted) before a full restart. After restart the counter restarts at 1 and IDs may be reused. Please verify that no sink or checkpoint storage path (e.g. file names keyed by checkpointId) can collide with leftover state from the aborted attempts, or clean them up on restore. Please also make the exclusion consistent with how engine_runningJobMetrics and engine_checkpoint_monitor are handled in #13013 (same mechanism/config location), and add a short note in the Zeta checkpoint/IMap persistence docs listing which engine* maps are intentionally not persisted and why. <!-- streview-comment:2098 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
