nchammas commented on pull request #31742: URL: https://github.com/apache/spark/pull/31742#issuecomment-797624636
> To have checkpoint directory cleaned up at shutdown one can register the directory to be deleted at exit: > > ``` > FileSystem fs = FileSystem.get(conf); > fs.deleteOnExit(checkpointPath); > ``` Are you suggesting I do that somewhere in this PR, or are you saying this is the workaround we should recommend to people who actually want checkpoints to be cleaned up automatically on shutdown? If the former, then I am confused. This goes against your argument that we should not do this automatic cleanup on shutdown. So I think you mean the latter. If so, I'm not sure where we would document that suggestion for users. Every language will require a different technique, so I don't think it makes sense to add to the docs for `.checkpoint()`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
