Deependra-Patel opened a new pull request, #42391: URL: https://github.com/apache/spark/pull/42391
### What changes were proposed in this pull request? Deleting files on executor exit when migration is enabled ### Why are the changes needed? 1. This will release extra disk space on the node reducing the failures due to disk running out of space 2. If there is no shuffle data then YARN can decommission the node right away. Needs this other PR to be merged to expose this metric to YARN https://github.com/apache/spark/pull/42071 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested manually. Will add unit tests once we get traction on this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
