Ngone51 commented on pull request #33477: URL: https://github.com/apache/spark/pull/33477#issuecomment-884712661
> Once a job in a Spark application completes, the shuffle files are cleaned up by the executors. Actually, it's cleaned up by the `ContextCleaner` when the shuffle dependency can be GCed. Job completion doesn't ensure that. Could you update the description to make it more accurate? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
