WeichenXu123 commented on a change in pull request #25315:
[SPARK-28582][PYSPARK] Fix pyspark daemon exit failed when receive SIGTERM on
Python 3.7
URL: https://github.com/apache/spark/pull/25315#discussion_r309503553
##########
File path: python/pyspark/daemon.py
##########
@@ -102,7 +102,7 @@ def shutdown(code):
signal.signal(SIGTERM, SIG_DFL)
# Send SIGHUP to notify workers of shutdown
os.kill(0, SIGHUP)
- sys.exit(code)
+ os._exit(code)
Review comment:
This line `os.kill(0, SIGHUP)` kill child processes by sending SIGHUP to
them.
so looks like calling sys.exit is the last thing to do in daemon. We have
already done necessary clean before calling exit.
So I prefer `os._exit` rather than `sys.exit` here, because `sys.exit` will
throw `SystemExit` exception in main thread. If this `SystemExit` exception is
swallowed by user code somewhere or in finally block run into some blocking
code, then the kill will fail.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
