[issue27081] Cannot capture sys.stderr output from an uncaught exception in a multiprocessing Process using a multiprocessing Queue

[Davin Potts]

Davin Potts added the comment:

The spawned process (you appear to have run on Windows, so I'm assuming spawn 
but that's not so significant) has triggered an unhandled exception.  If the 
triggering and subsequent sending of the traceback to stderr is synchronous and 
completes before Python actually kills the process, then the original "one 
would expect" statement would make sense.  Removing the use of the Queue 
entirely and substituting the use of a socket to communicate from the child 
(spawned) process to some distinct listener process results in only a portion 
of the traceback text being received by the listener (or sometimes little to 
none of the traceback text).  This suggests that the original premise is false 
and the expectation inaccurate.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27081] Cannot capture sys.stderr output from an uncaught exception in a multiprocessing Process using a multiprocessing Queue

[ppperry]

ppperry added the comment:

This issue isn't specific to recursion errors. It only occurs when the error 
message is long enough, so #26823 would fix the RecursionError case, but it 
would still happen when someone calls a function with a billion-character-long 
name that raises an error, for example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27081] Cannot capture sys.stderr output from an uncaught exception in a multiprocessing Process using a multiprocessing Queue

[Emanuel Barry]

Emanuel Barry added the comment:

Worth of note: #26823 will actually change printing to stderr for recursion 
errors (if/when it gets merged that is; I don't know what's holding it up). It 
might be interesting to see if you can still witness this issue with the patch 
applied.

--
nosy: +ebarry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27081] Cannot capture sys.stderr output from an uncaught exception in a multiprocessing Process using a multiprocessing Queue

[ppperry]

ppperry added the comment:

I believe that regardless of the number of prints to sys.stderr that happen 
before the recursion error, all of them will get sent to the parent. The 
problem is that the queue is flushed before the uncaught error is sent to 
stderr, not after.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27081] Cannot capture sys.stderr output from an uncaught exception in a multiprocessing Process using a multiprocessing Queue

[Davin Potts]

Davin Potts added the comment:

I took the example snippet and cleaned things up a bit, adding some crude 
timestamping and commentary.  (Attached)

In the example, when the Process finally dies, a lot of information has been 
put onto the Queue but it hasn't necessarily had enough time to be synced to 
the parent process so it may be only a truncated subset of the data that 
remains available/visible/retrievable on the parent.  How much data gets synced 
to the parent depends on many factors and should even vary from one run to 
another.

That is to say that a multiprocessing.Queue is not going to hold up a dying 
process to finish communicating data to other processes.

--
Added file: http://bugs.python.org/file43613/issue27081.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27081] Cannot capture sys.stderr output from an uncaught exception in a multiprocessing Process using a multiprocessing Queue

[ppperry]

Changes by ppperry :


--
title: Multiprocessing is not robust against sys.stderr changes involving 
queues -> Cannot capture sys.stderr output from an uncaught exception in a 
multiprocessing Process using a multiprocessing Queue

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com