New submission from Aaron Halfaker:
multiprocessing.imap will hang and not raise an error if an error occurs in the
generator that is being mapped over. I'd expect the error to be raised and/or
the process to fail.
For example, run the following code in python 2.7 or 3.4:
from multiprocessing import Pool
def add_one(v):
return v+1
pool = Pool(processes=2)
values = ["1", "2", "3", "4", "foo", "5", "6", "7", "8"]
value_iter = (int(v) for v in values)
for new_val in pool.imap(add_one, value_iter):
print(new_val)
And output should look something like this:
$ python demo_hanging.py
2
3
4
5
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.4/multiprocessing/pool.py", line 378, in
_handle_tasks
for i, task in enumerate(taskseq):
File "/usr/lib/python3.4/multiprocessing/pool.py", line 286, in <genexpr>
self._taskqueue.put((((result._job, i, func, (x,), {})
File "demo_hanging.py", line 9, in <genexpr>
value_iter = (int(v) for v in values)
ValueError: invalid literal for int() with base 10: 'foo'
The script will then hang indefinitely.
----------
components: Library (Lib)
messages: 260032
nosy: Aaron Halfaker
priority: normal
severity: normal
status: open
title: Multiprocessing imap hangs when generator input errors
type: behavior
versions: Python 2.7, Python 3.4
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26333>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
