Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/9946#issuecomment-173298943
Just to point out here I may re-open this. I would still rather fix this,
a known bug that I can reproduce then worry about a theoretical it might break
something. Unless of course someone can give me a hard use case that this
breaks.
At the point we call System.exit here all user code is done and we are
terminating. If there is something you know of that the user should be allowed
to do after this then we should create something like a shutdown hook so the
user can properly clean it up.
A timeout would probably work here but I'm not fond of it here, it is going
to delay it for everyone and you don't give the resources back as quickly. You
could also end up in the same situation where if you choose a timeout, say 3
seconds and if this theoretical user code didn't finish within that 3 seconds
you would kill it anyway.
Actually thinking about this more, maybe this is a perfect time as we can
put it in 2.0 so change in behavior is more reasonable. thoughts ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
