[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
Alon Diamant added the comment: The patches I attached do 2 things: 1. A deadlock is prevented, wherein the main thread waits forever for the Pool thread/s to finish their execution, while they wait for instructions to terminate from the _task_handler thread which has died. Instead, the exception are caught and handled and termination of the pool execution is performed. 2. The exception that was raised is caught and passed to the main thread, and is re-thrown in the context of the main thread - hence the user catch it and handle it, or - at the very least - be aware of the issue. I tested the patch to the best of my abilities, and am almost certain nothing was changed performance wise nor anything broken. Further eyes would, of course, only help for confirming this. -- ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
Changes by Alon Diamant : Added file: http://bugs.python.org/file37453/Issue_23051_fix_v3_4.patch ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
Changes by Alon Diamant : Removed file: http://bugs.python.org/file37452/Issue_23051_fix_v3_4.patch ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
Changes by Alon Diamant : Added file: http://bugs.python.org/file37452/Issue_23051_fix_v3_4.patch ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
Changes by Alon Diamant : Added file: http://bugs.python.org/file37451/Issue_23051_reproducer_v3_4.py ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
Changes by Alon Diamant : -- keywords: +patch Added file: http://bugs.python.org/file37450/Issue_23051_fix_v2_7.patch ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
Changes by Alon Diamant : Added file: http://bugs.python.org/file37449/Issue_23051_reproducer_v2_7.py ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23051] multiprocessing.pool methods imap() and imap_unordered() cause deadlock
New submission from Alon Diamant: When imap() or imap_unordered() are called with the iterable parameter set as a generator function, and when that generator function raises an exception, then the _task_handler thread (running the method _handle_tasks) dies immediately, without causing the other threads to stop and without reporting the exception to the main thread (that one that called imap()). I saw this issue in Python 2.7.8, 2.7.9 and 3.4.2. Didn't check other versions, but I assume this is a bug in all Python versions since 2.6. I will be attaching examples that reproduce this issue, as well as patches for both Python 2.7 and Python 3.4. -- components: Library (Lib) messages: 232643 nosy: advance512 priority: normal severity: normal status: open title: multiprocessing.pool methods imap() and imap_unordered() cause deadlock type: behavior versions: Python 2.7, Python 3.4 ___ Python tracker <http://bugs.python.org/issue23051> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com