attilapiros commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-638684678
> So I don't want to stop the executor directly once the block migration is done. Instead, I have a follow-up JIRA which I've started working on that shutdowns the executor once the block migration has completed and there are no running tasks. I think it's ok to (temporarily) leak threads since decommissioning (as triggered currently) is only in the situation where the executor will be exiting soon anyways. I understand your point but as we are testing the decommissioning these threads are leaking during testing and even aggregating by each test using this feature. We are logging this leaks in this method https://github.com/apache/spark/blob/master/core/src/test/scala/org/apache/spark/ThreadAudit.scala#L106 at the end of the suite. I believe it is better to keep such log completely clean if the required code change is easy otherwise the next person felt to be encouraged to add just one more violation. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
