[
https://issues.apache.org/jira/browse/FLINK-30513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698192#comment-17698192
]
Weijie Guo edited comment on FLINK-30513 at 3/9/23 6:19 PM:
------------------------------------------------------------
master(1.18) via 13779ab8e4f5539ca311d9f233d031d818af6450.
revert via 593cc139ab30bb81ab38d94b7b697d00eaaecada as [comments in pull
request|https://github.com/apache/flink/pull/21673#discussion_r1130781417].
was (Author: weijie guo):
master(1.18) via 13779ab8e4f5539ca311d9f233d031d818af6450.
release-1.17 waiting for CI.
> HA storage dir leaks on cluster termination
> --------------------------------------------
>
> Key: FLINK-30513
> URL: https://issues.apache.org/jira/browse/FLINK-30513
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.15.0, 1.16.0
> Reporter: Zhanghao Chen
> Assignee: Zhanghao Chen
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-12-27-21-32-17-510.png
>
>
> *Problem*
> We found that HA storage dir leaks on cluster termination for a Flink job
> with HA enabled. The following picture shows the HA storage dir (here on
> HDFS) of the cluster czh-flink-test-offline (of application mode) after
> canelling the job with flink-cancel. We are left with an empty dir, and too
> many empty dirs will greatly hurt the stability of HDFS NameNode!
> !image-2022-12-27-21-32-17-510.png|width=582,height=158!
>
> Furthermore, in case the user choose to retain the checkpoints on job
> termination, we will have the completedCheckpoints leaked as well. Note that
> we no longer need the completedCheckpoints files as we'll directly recover
> retained CPs from the CP data dir.
> *Root Cause*
> When we run AbstractHaServices#closeAndCleanupAllData(), we cleaned up blob
> store, but didn't clean the HA storage dir.
> *Proposal*
> Clean up the HA storage dir after cleaning up blob store in
> AbstractHaServices#closeAndCleanupAllData().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
