[
https://issues.apache.org/jira/browse/FLINK-33288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776064#comment-17776064
]
Xin Chen edited comment on FLINK-33288 at 10/17/23 8:43 AM:
------------------------------------------------------------
Thank you very much! [~Zhanghao Chen]. I have hardly heard of
'job-result-store', and I am not quite sure what scenario this parameter is
specifically used for. Do you know which version of Flink that was added?
Because we used Flink 1.12.2 before, we couldn't find this parameter in it, but
we can saw it in 16 and 17.
And directory leak also exists in 'job-result-store.storage-path' in Flink
1.16-1.17:
!image-2023-10-17-16-43-07-859.png!
was (Author: JIRAUSER298666):
Thank you very much! [~Zhanghao Chen]. I have hardly heard of
'job-result-store', and I am not quite sure what scenario this parameter is
specifically used for. Do you know which version of Flink that was added?
Because we used Flink 1.12.2 before, we couldn't find this parameter in it, but
we can saw it in 16 and 17.
> Empty directory residue with appid name in HA(highly-available) related
> directory of hdfs, not cleaned
> ------------------------------------------------------------------------------------------------------
>
> Key: FLINK-33288
> URL: https://issues.apache.org/jira/browse/FLINK-33288
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Configuration
> Affects Versions: 1.16.2, 1.17.1
> Reporter: Xin Chen
> Priority: Major
> Attachments: image-2023-10-17-16-43-07-859.png, screenshot-1.png,
> screenshot-2.png
>
>
> When I submitted a large number of tasks in Flink-on-Yarn mode and
> successfully executed, I unexpectedly found a large number of empty
> directories left in the directory related to 'high availability.storageDir'
> on hdfs, with appids as shown below. I believe this must be cleared! However,
> after verification in the environments of 1.16.2 and 1.17.1, it was proven
> that neither of them solved this problem.
> my flink-conf.yaml about 'high availability.storageDir':
> {code:java}
> high-availability.storageDir: hdfs://hdfsHACluster/flink/recovery
> {code}
> !screenshot-1.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
