Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9946#issuecomment-161111303 But isn't the scenario here that the user app isn't done, because it spawned non-daemon threads that are doing something? I agree it's not good practice, but if apps avoided doing this entirely we'd have no problem to begin with. The question is what to do if such a user thread does exist and runs long. Killing it immediately could cause problems, and it's not terribly theoretical: imagine persisting data to disk or writing to a socket and failing to write all the bytes. Not-killing it of course opens the possibility that the thread never stops at all. Which is worse? The long-running user thread is the app's "fault" and is pretty easy to debug by looking at a stack dump. On the other hand killing threads straight away could cause problems for a sort of reasonably behaving app. (Also imagine this non-daemon thread could be in library code.) Maybe some kind of timeout mostly mitigates the issue, but then I think it's just trying to save a fairly clearly misbehaving app from itself at a non-trivial cost. You're always going to have the possibility of a stuck process (deadlock, infinite loop in a driver, etc) and need to be able to kill that if needed as an admin.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org