[issue13831] get method of multiprocessing.pool.Async should return full traceback
Roundup Robot added the comment: New changeset a2928dd2fde4 by Richard Oudkerk in branch 'default': Correct issue number for c4f92b597074 in Misc/NEWS from #13813 to #13831 http://hg.python.org/cpython/rev/a2928dd2fde4 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
Changes by Richard Oudkerk shibt...@gmail.com: -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
Richard Oudkerk added the comment: The relevant changeset was c4f92b597074, but I wrote the wrong issue number in the commit message and Misc/NEWS. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
Richard Oudkerk added the comment: Attached is a patch for 3.4 which uses the __cause__ hack to embed the remote traceback in the local traceback. It will not work for 2.x though. import multiprocessing, subprocess with multiprocessing.Pool() as p: p.apply(subprocess.Popen, (1,)) ... multiprocessing.pool.RemoteTraceback: Traceback (most recent call last): File /home/oudkerk/Repos/py-default/Lib/multiprocessing/pool.py, line 114, in worker result = (True, func(*args, **kwds)) File /home/oudkerk/Repos/py-default/Lib/subprocess.py, line 838, in __init__ restore_signals, start_new_session) File /home/oudkerk/Repos/py-default/Lib/subprocess.py, line 1317, in _execute_child args = list(args) TypeError: 'int' object is not iterable The above exception was the direct cause of the following exception: Traceback (most recent call last): File stdin, line 1, in module File /home/oudkerk/Repos/py-default/Lib/multiprocessing/pool.py, line 245, in apply return self.apply_async(func, args, kwds).get() File /home/oudkerk/Repos/py-default/Lib/multiprocessing/pool.py, line 588, in get raise self._value TypeError: 'int' object is not iterable -- keywords: +patch Added file: http://bugs.python.org/file30109/pool-traceback.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
Richard Oudkerk added the comment: It might be possible to come up with a hack so that when the exception is unpickled in the main process it gets a secondary exception chained to it using __cause__ or __context__ whose stringification contains the stringification of the original traceback. -- versions: +Python 3.4 -Python 2.6, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
Richard Oudkerk added the comment: Pickling an exception (assuming it works) does not capture the traceback. Doing so would be difficult/impossible since the traceback refers to a linked list of frames, and each frame has references to lots of other stuff like the code object, the global dict, local dict, builtin dict, ... I certainly do not know how to make traceback objects pickle compatible. But you could wrap any exception raised by your function in another exception whose representation contains a formatted traceback of the original exception. E.g. class WrapException(Exception): def __init__(self): exc_type, exc_value, exc_tb = sys.exc_info() self.exception = exc_value self.formatted = ''.join(traceback.format_exception(exc_type, exc_value, exc_tb)) def __str__(self): return '%s\nOriginal traceback:\n%s' % (Exception.__str__(self), self.formatted) def go(): try: 1/0 except Exception: raise WrapException() Then raising an unpickled WrapException instance gives the original traceback try: go() ... except Exception as e: exc = e ... raise pickle.loads(pickle.dumps(exc)) Traceback (most recent call last): File stdin, line 1, in module __main__.WrapException: Original traceback: Traceback (most recent call last): File stdin, line 3, in go ZeroDivisionError: integer division or modulo by zero -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
Changes by Paul Winkler pw_li...@slinkp.com: -- nosy: +slinkp ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
Changes by Paul Winkler pw_li...@slinkp.com: -- versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13831] get method of multiprocessing.pool.Async should return full traceback
New submission from Faheem Mitha fah...@faheem.info: The documentation in http://docs.python.org/library/multiprocessing.html#module-multiprocessing.pool says class multiprocessing.pool.AsyncResult¶ The class of the result returned by Pool.apply_async() and Pool.map_async(). get([timeout]) Return the result when it arrives. If timeout is not None and the result does not arrive within timeout seconds then multiprocessing.TimeoutError is raised. If the remote call raised an exception then that exception will be reraised by get(). Consider the example code from multiprocessing import Pool def go(): print 1 raise Exception(foobar) print 2 p = Pool() x = p.apply_async(go) x.get() p.close() p.join() ### The traceback from this is Traceback (most recent call last): File stdin, line 10, in module File /usr/lib/python2.6/multiprocessing/pool.py, line 422, in get raise self._value Exception: foobar 1 As is clear in this example, this is not a full traceback - it only shows the traceback to the line where get is located and gives no further information. This is the case in all the other places I have used get. It seems to me that it *should* return the full traceback, which may contain important information missing in such a partial one. I don't know whether one would call this a feature request or a bug report. Maybe there is some technical reason why this is not possible, but I can't think of one. -- components: Library (Lib) messages: 151651 nosy: fmitha priority: normal severity: normal status: open title: get method of multiprocessing.pool.Async should return full traceback type: enhancement versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13831 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
