New submission from Selim Belhaouane <selim.belhaou...@gmail.com>: Cannot clear signal handler set with loop.add_signal_handler in forked process with signal.signal
# Context I'm running an async web server with uvicorn[1] and have background processes (using multiprocessing) doing CPU-bound work. uvicorn install signal handlers with loop.add*signal_handler, if available[2]. It also implements the common "attempt graceful shutdown on the first signal, and forcefully shutdown on the second (or more) signal" pattern. The problem I noticed was: when \_forking* at least one process, the server never gracefully shuts down, **even if I install new signal handlers in the subprocesses**. # Problem Signal handlers installed with loop.add*signal_handler cannot be cleared in forked processes with `signal.signal` (see Experiment 1), \_unless* using `signal.SIG_DFL` or `signal.SIG_IGN` in the forked processes (see Experiment 3). For the record: - When installing signal handlers in the parent process with signal.signal, this is not a problem (see Experiment 2). - When using multiprocessing with the "spawn" method, this is not a problem. Unsurprising, since signal handlers are not inherited from the parent process when using "spawn" (see Experiment 4). - When install signal handlers in the child proceess with loop.add_signal_handler, this is not a problem (see Experiment 5). # Experiments You'll find a minimal `exp.py` file in the attached archive. There's a few tweakable parameters at the top. I just run this with `python3.X exp.py`, wait a second or two, and hit Ctrl+C. `results.txt` shows the results of a few experiments with different set of parameters. Note that the point at which I hit Ctrl+C is indicated by the ♥ (heart) symbol, due to some terminal weirdness, although this is quite useful in this case! The subsections below detail what happens when Ctrl+C is hit. You guys probably know this already, but Ctrl+C basically sends SIGINT to all processes in the process tree. ## Experiment 1 - Installs a signal handler in the parent process with loop.add_signal_handler - Starts 3 subprocesses with "fork" - The subprocesses install a new signal handler with signal.signal Outcome: both the parent handler and child handlers get called (3 calls to handle_sig_worker, 4 calls to handle_sig_main) **Expected**: I would expect a single call to handle_sig_main, and 3 calls to handle_sig_worker, as in Experiment 2 (which uses signal.signal instead of loop.add_signal_handler) or Experiment 4 (which uses "spawn" instead of "fork") or Experiment 5 () ## Experiment 2 As Experiment 1, but signal handlers are installed in the parent process with signal.signal Outcome: the parent handler gets called once, and the child handlers get called 3 times ## Experiment 3 Same as Experiment 1, but this time using `signal.SIG_DFL` as the callback in the child processes. Outcome: the child processes immediately terminate, thanks to SIG_DFL, and the parent handler gets called only once! ## Experiment 4 Same as Experiment 1, but this time using "spawn" as the multiprocessing start method. Outcome: same as Experiment 2: parent handlers gets called once, child handlers get called 3 times ## Experiment 5 Same as Experiment 1, but this time installing signal handlers in the child processes with loop.add_signal_handler. Outcome: Same as Experiment 2. # Environment I was able to replicate the issue with python 3.7, 3.8, 3.9 and 3.10 (didn't even try 3.6) on Ubuntu 20.04. I obtained python via `apt`. [1]: https://www.uvicorn.org/ [2]: https://github.com/encode/uvicorn/blob/61a6cabb4580e1c923df396eac264803f599412c/uvicorn/server.py#L281 ---------- components: asyncio files: python-async-signal-bug.zip messages: 385928 nosy: asvetlov, selimb, yselivanov priority: normal severity: normal status: open title: Impossible to override signal handler set with add_signal_handler in forked process type: behavior versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9 Added file: https://bugs.python.org/file49774/python-async-signal-bug.zip _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue43064> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
