New submission from Michael:
Following code snippet causes a deadlock on Linux:
"""
import multiprocessing.pool
import signal
def signal_handler(signum, frame):
pass
if __name__ == '__main__':
signal.signal(signal.SIGTERM, signal_handler)
pool = multiprocessing.pool.Pool(processes=1)
pool.terminate() # alternatively - raise Exception("EXCEPTION")
"""
The reason is that the termination code starts before the worker processes
being fully initialized.
Here, parent process acquires a forever-lock:
"""
@staticmethod
def _help_stuff_finish(inqueue, task_handler, size):
# task_handler may be blocked trying to put items on inqueue
util.debug('removing tasks from inqueue until task handler finished')
inqueue._rlock.acquire() < -----------------
while task_handler.is_alive() and inqueue._reader.poll():
inqueue._reader.recv()
time.sleep(0)
"""
And then the worker processes are getting stuck here:
"""
def worker(...):
while maxtasks is None or (maxtasks and completed < maxtasks):
try:
task = get() < ----------------- trying to acquire
the same lock
except (EOFError, OSError):
util.debug('worker got EOFError or OSError -- exiting')
break
"""
Whats going on then? As far as the default process start method is set to
'fork', worker subprocesses inherit parent's signal handler. Trying to
terminate workers from _terminate_pool() doesn't have any effect. Finally,
processes enter into a deadlock when parent join()-s workers.
----------
components: Library (Lib)
messages: 289248
nosy: mapozyan
priority: normal
severity: normal
status: open
title: Deadlock in multiprocessing.pool.Pool on terminate
versions: Python 3.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29759>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
