StefanRRichter commented on a change in pull request #7942:
[FLINK-11696][checkpoint] Avoid to send mkdir requests to DFS from task side
URL: https://github.com/apache/flink/pull/7942#discussion_r264284886
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FsCheckpointStorage.java
##########
@@ -107,6 +105,13 @@ public CheckpointStorageLocation
initializeLocationForCheckpoint(long checkpoint
// prepare all the paths needed for the checkpoints
final Path checkpointDir =
createCheckpointDirectory(checkpointsDirectory, checkpointId);
+ if (!areDirectoriesCreated) {
+ fileSystem.mkdirs(checkpointsDirectory);
+ fileSystem.mkdirs(sharedStateDirectory);
Review comment:
Now that only shared and taskowned would be left, I wonder if it would not
be cleaner to just create them in an init method that is only called on the
JM/checkpoint coordinator side. Do you think there is still benefit in true
lazyness, like only create the dirs when the first checkpoint is actually
triggered. Or is it sufficient if not all TMs will also make those calls? From
code simplicity, it would be better if we can avoid any lazy initialization
flags.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
