[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

Change by Ionel Cristian Mărieș : -- nosy: +ionelmc ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

Michael added the comment: I found a couple of other cases when deadlock still occurs. 1. _help_stuff_finish may remove sentinels from the queue. Some of the workers will then never get a signal to terminate. 2. worker handler thread may be terminated too late, so it may spawn new workers

[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

Michael added the comment: If `task_handler._state = TERMINATE` is done before call to _help_stuff_finish(), then the following loop `while task_handler.is_alive() and inqueue._reader.poll()` in that function won't work as `is_alive()` will obviously return False. --

[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

Antoine Pitrou added the comment: This is tricky to reproduce but can definitely happen. It seems it is enough to release the lock when done, why did you have to move the `task_handler._state = TERMINATE` line in your patch? -- nosy: +pitrou ___

[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

Changes by Xiang Zhang : -- nosy: +davin versions: -Python 3.3, Python 3.4 ___ Python tracker ___

[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

Changes by Michael : -- type: -> behavior versions: +Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6 ___ Python tracker

[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

Michael added the comment: This patch kind of solves the issue. Not a nice one, but perhaps the safest one. https://github.com/michael-a-cliqz/cpython/commit/1536c8c8cfc5a87ad4ab84d1248cb50fefe166ae -- ___ Python tracker

[issue29759] Deadlock in multiprocessing.pool.Pool on terminate

New submission from Michael: Following code snippet causes a deadlock on Linux: """ import multiprocessing.pool import signal def signal_handler(signum, frame): pass if __name__ == '__main__': signal.signal(signal.SIGTERM, signal_handler) pool =