[
https://issues.apache.org/jira/browse/FLINK-30412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17648325#comment-17648325
]
Xintong Song commented on FLINK-30412:
--------------------------------------
[~xiaodao],
Thanks for the explanation. Now I get your point.
I think the problem is that, in {{StreamingJobGraphGenerator}} we use `interval
== Long.MAX_VALUE` to indicate disabling checkpointing, while in
{{ExecutionGraphBuilder}} we use `snapshotSettings == null`. This inconsistency
will indeed result in creating of empty directories when checkpointing is
disabled.
[~gaoyunhaii], could you help double-check on this?
> create many checkpoint empty dir when job not enable checkpoint
> ---------------------------------------------------------------
>
> Key: FLINK-30412
> URL: https://issues.apache.org/jira/browse/FLINK-30412
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
> Affects Versions: 1.12.7, 1.13.6, 1.15.2
> Reporter: xiaodao
> Priority: Major
>
> when we submit job to flink session cluster , after a long time, we find it
> create too much
> empty checkpoint dir,and it over hdfs max node limit ;
> i found StreamingJobGraphGenerator set snapshot whennever the job is open
> checkpoint;
> jobGraph.setSnapshotSettings(settings)
> {code:java}
> private void configureCheckpointing()
> CheckpointConfig cfg = streamGraph.getCheckpointConfig(); long interval =
> cfg.getCheckpointInterval(); if (interval < MINIMAL_CHECKPOINT_TIME) { //
> interval of max value means disable periodic checkpoint interval =
> Long.MAX_VALUE; }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
