[
https://issues.apache.org/jira/browse/FLINK-13856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915667#comment-16915667
]
Stephan Ewen commented on FLINK-13856:
--------------------------------------
In general, that is a good idea.
However, this is an optimization that we can do only for exclusive state
(belongs only to one checkpoint) but not shared state (par of incremental
checkpoints).
I remember wanting to do that and there was some issue with selectively
deleting the shared state first, and then dropping the exclusive location as a
whole.
Furthermore, this only works on proper file systems, not on object stores like
S3.
> Reduce the delete file api when the checkpoint is completed
> -----------------------------------------------------------
>
> Key: FLINK-13856
> URL: https://issues.apache.org/jira/browse/FLINK-13856
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.8.1, 1.9.0
> Reporter: andrew.D.lin
> Priority: Major
> Attachments: f6cc56b7-2c74-4f4b-bb6a-476d28a22096.png
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When the new checkpoint is completed, an old checkpoint will be deleted by
> calling CompletedCheckpoint.discardOnSubsume().
> When deleting old checkpoints, follow these steps:
> 1, drop the metadata
> 2, discard private state objects
> 3, discard location as a whole
> In some cases, is it possible to delete the checkpoint folder recursively by
> one call?
> As far as I know the full amount of checkpoint, it should be possible to
> delete the folder directly.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
