HyukjinKwon commented on a change in pull request #25315:
[SPARK-28582][PYSPARK] Fix pyspark daemon exit failed when receive SIGTERM on
Python 3.7
URL: https://github.com/apache/spark/pull/25315#discussion_r309484449
##########
File path: python/pyspark/daemon.py
##########
@@ -102,7 +102,7 @@ def shutdown(code):
signal.signal(SIGTERM, SIG_DFL)
# Send SIGHUP to notify workers of shutdown
os.kill(0, SIGHUP)
- sys.exit(code)
+ os._exit(code)
Review comment:
If this is really a bug in Python 3.7, let's file a bug in
https://bugs.python.org/ , confirm it was a bug , and use this workaround only
for Python 3.7 conditionally. Ideally, we should prepare a minimal reproducer
and share there.
Lastly, is this test-only issue or actual problem with Python 3.7 (daemon
and/or workers are not being dead)? If so, this issue is a critical issue that
leaks resources.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
