[
https://issues.apache.org/jira/browse/FLINK-34117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu resolved FLINK-34117.
--------------------------------
Fix Version/s: 1.18.2
Resolution: Fixed
Fixed via
master: 1d0f34e5a4a9127724bd09ecd0d9c0c0dba431f3
1.18: f287fb8f3c89cad1db028d8119c748b1c7b2194b
> CompactCoordinator for table file sink loses data upon job termination
> ----------------------------------------------------------------------
>
> Key: FLINK-34117
> URL: https://issues.apache.org/jira/browse/FLINK-34117
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Affects Versions: 1.18.1
> Reporter: Alexander Fedulov
> Assignee: Alexander Fedulov
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.0, 1.18.2
>
>
> CompactCoordinator accumulates data in currentInputFiles and only rolls them
> into inputFiles in snapshotState(). At the same time it relies on separately
> receiving checkpoint indications from the upstream operator via
> processElement() (EndCheckpoint). If the job terminates, the final
> EndCheckpoint can arrive before the snapshotState() gets called. This leads
> to data loss (all events in currentInputFiles get discarded).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
