afedulov opened a new pull request, #24195: URL: https://github.com/apache/flink/pull/24195
`CompactCoordinator` accumulates data in `currentInputFiles` and only rolls them into `inputFiles` when `snapshotState()` gets called. At the same time it relies on separately receiving checkpoint indications from the upstream operator via `processElement()` (`EndCheckpoint`). If the job terminates, the final `EndCheckpoint` can arrive before the `snapshotState()` gets called. This leads to data loss (all events in `currentInputFiles` get discarded). This is a backport from the 1.19 branch: https://github.com/apache/flink/pull/24120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
