ericm-db commented on PR #56907:
URL: https://github.com/apache/spark/pull/56907#issuecomment-4859605283
Good catch, fixed in bd938e7.
The timeout path now signals the whole process group and escalates to
SIGKILL if a graceful stop doesn't take, so the child JVM is reaped rather than
orphaned:
```python
if os.name == "posix":
os.killpg(os.getpgid(proc.pid), signal.SIGTERM)
else:
proc.terminate()
try:
proc.wait(timeout=10)
except Exception:
if os.name == "posix":
os.killpg(os.getpgid(proc.pid), signal.SIGKILL)
else:
proc.kill()
```
This works because the daemon is a session leader (`start_new_session=True`)
and pyspark launches the JVM without its own new process group
(`java_gateway.py` only sets a `preexec_fn` to ignore SIGINT), so the JVM stays
in the daemon's group and `killpg` reaps both.
I kept the daemon's own SIGTERM handler where it is (after `getOrCreate()`),
since the group-kill + SIGKILL escalation covers the pre-handler window
regardless. Verified manually: started a daemon, waited until it had spawned
its JVM child, then ran the terminate helper — both the daemon and the JVM are
gone afterward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
