Github user BryanCutler commented on the pull request:
https://github.com/apache/spark/pull/11746#issuecomment-205451978
Hi @zsxwing , thanks for checking this out! I made a couple changes to fix
the race condition you pointed out. The shutdown hook now calls `kill()` which
will kill the process and stop the thread cleanly. Killing the thread is
synchronized with the process starting. So if `kill()` is called before the
process starts, then then thread will be blocked from attempting to start the
process and will exit.
I tested this by exiting the JVM at various points in the DriverRunner
usage and it seems to work well (I disabled the `WorkerWatcher` during test to
make sure the hook was terminating the process).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
