StephanEwen commented on issue #10768: [FLINK-15012][checkpoint] Introduce shutdown to CheckpointStorageCoordinatorView to clean up checkpoint directory. URL: https://github.com/apache/flink/pull/10768#issuecomment-594422896 Should we make the directory cleanup **NON recursive**? That might be the safer option. Given that we only want the parent/container directories removed, non-recursive delete should be fine. If something is still in those directories, it may be unsafe to delete them. Maybe a retained checkpoint from a previous job? Maybe an incremental safepoint in the future? It may be dangerous to assume that at shutdown time, all data in these directories can be deleted. If we see left-over data because checkpoints failed to clean up properly, then the shutdown part may not be the right place to clean this up, because it has too little information to know if this truly safe to be deleted. For that we need some "cleanup safety nets" that understand which shared files are still referenced, which checkpoints have been discarded (up to which point).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
