[
https://issues.apache.org/jira/browse/FLINK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810848#comment-17810848
]
Alexander Fedulov commented on FLINK-34117:
-------------------------------------------
[~leonard] thanks! Here is the backport PR
https://github.com/apache/flink/pull/24195
> CompactCoordinator for table file sink loses data upon job termination
> ----------------------------------------------------------------------
>
> Key: FLINK-34117
> URL: https://issues.apache.org/jira/browse/FLINK-34117
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Affects Versions: 1.18.1
> Reporter: Alexander Fedulov
> Assignee: Alexander Fedulov
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.0
>
>
> CompactCoordinator accumulates data in currentInputFiles and only rolls them
> into inputFiles in snapshotState(). At the same time it relies on separately
> receiving checkpoint indications from the upstream operator via
> processElement() (EndCheckpoint). If the job terminates, the final
> EndCheckpoint can arrive before the snapshotState() gets called. This leads
> to data loss (all events in currentInputFiles get discarded).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
