[ https://issues.apache.org/jira/browse/FLINK-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-5086: ---------------------------------- Labels: auto-deprioritized-major (was: stale-major) > Clean dead snapshot files produced by the tasks failing to acknowledge > checkpoints > ---------------------------------------------------------------------------------- > > Key: FLINK-5086 > URL: https://issues.apache.org/jira/browse/FLINK-5086 > Project: Flink > Issue Type: Bug > Components: Runtime / Checkpointing > Reporter: Xiaogang Shi > Priority: Major > Labels: auto-deprioritized-major > > A task may fail when performing checkpoints. In that case, the task may have > already copied some data to external storage. But since the task fails to > send the state handler to {{CheckpointCoordinator}}, the copied data will not > be deleted by {{CheckpointCoordinator}}. > I think we must find a method to clean such dead snapshot data to avoid > unlimited usage of external storage. > One possible method is to clean these dead files when the task recovers. When > a task recovers, {{CheckpointCoordinator}} will tell the task all the > retained checkpoints. The task then can scan the external storage to delete > all the snapshots not in these retained checkpoints. -- This message was sent by Atlassian Jira (v8.3.4#803005)