[GitHub] Myasuka commented on issue #7281: [FLINK-11107][state] Avoid memory stateBackend to create arbitrary folders under HA path when no checkpoint path configured

[GitBox]

Myasuka commented on issue #7281: [FLINK-11107][state] Avoid memory 
stateBackend to create arbitrary folders under HA path when no checkpoint path 
configured
URL: https://github.com/apache/flink/pull/7281#issuecomment-468532245
 
 
   Since Flink-1.8 is about to release, @StephanEwen @StefanRRichter could 
anyone take a look at this problem?
   I submitted the same job with the same configuration (no checkpoint path but 
HA configured) with released Flink-1.3.2 (still has no such 
`MemoryStateBackend` creating random checkpoint path code, which should be 
treated as `old behavior`) and Flink-1.7.2 (already contained that part of 
code.)
   
   As you can see `Flink-1.3.2` would have a blob service folder, a completed 
checkpoint file and a submitted job graph file. I think this is the `old 
behavior`.
   <img width="781" alt="20190301113237" 
src="https://user-images.githubusercontent.com/1709104/53614879-67353c80-3c16-11e9-8fac-0dee85b676d4.png";>
   
   However, `Flink-1.7.2` would have many checkpoint paths created by 
`MemoryStateBackend` from task-side, as you could guess, 
`41a7c8b8e62d81225868d2a5a60846f7` is the actual job-id of this job. These 
created checkpoint path should actually be useless, and might lead to 
`MaxDirectoryItemsExceededException` under high availability folder. 
   <img width="1052" alt="20190301113317" 
src="https://user-images.githubusercontent.com/1709104/53614943-9e0b5280-3c16-11e9-81c4-868c3187a09b.png";>
   Moreover, as you can see, I don't think this would `keep supporting the old 
behavior` due to the grate directory structure difference.
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services