[issue1230540] sys.excepthook doesn't work in threads

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

If we want to support low-level threads created by start_new_thread() we should 
call excepthook() from t_bootstrap instead of Thread._bootstrap_inner. 
Otherwise implementing excepthook() as a Thread method would be more convenient.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

It took 14 years, but this issue is now fixed ;-) I close it.

Python 3.8 beta 1 will be released in a few days with threading.excepthook(): 
please test it to ensure that it works as you expected.

As a follow-up, I created bpo-37069: "regrtest: log unraisable exceptions and 
uncaught thread exceptions".

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:


New changeset cd590a7cede156a4244e7cac61e4504e5344d842 by Victor Stinner in 
branch 'master':
bpo-1230540: Add threading.excepthook() (GH-13515)
https://github.com/python/cpython/commit/cd590a7cede156a4244e7cac61e4504e5344d842


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

> In any case, I think the namedtuple / structseq solution is elegant, because 
> we can add additional information later

I used the same design for the new sys.unraisablehook in bpo-36829 and I'm 
already working on adding a new 'err_msg' field to the argument passed to this 
took: PR 13488 :-)

I was trapped in the past when I had to modify warnings "hooks" 
(warnings.showwarning and warnings.formatwarning) when I had to add a new 
'source' parameter. I had to write wrappers which are fragile, to keep the 
backward compatibility.

> (the user must only be careful not to use tuple unpacking)

threading.excepthook doesn't mention the compatibility with tuple on purpose. 
It only documents attributes with their names.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

Antoine Pitrou:
> A Thread.excepthook() method does not allow to notify exceptions raised in 
> C-created threads ("dummy threads").

I modified my PR to use the threading identitifer (threading.get_ident()) as 
the thread name if args.thread is None.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

> Indeed, if you write your own Thread class, you can add a try...except
> in the Thread.run() method.  You don't need a dedicated
> Thread.excepthook() method.

Exactly. You can already do you best in your run() method to handle exceptions.

threading.excepthook is only there is everything else already failed.

FYI in my implementation, if threading.excepthook raises a new exception, it's 
also handled... by sys.excepthook this time ;-)


> The only way a per-thread hook could be useful is if you could set it
> *outside* of the Thread class (so not as a method), so that one can e.g.
> catch / report exceptions raised in threads launches by third-party
> libraries.

I discuss threading excepthook with Pablo and he asked me if it would be 
possible to have a different behavior depending if the thread is spawn by my 
application or by "third party code". Using threading.excepthook, you can mark 
your threads that you spawn directly using a specific name, a special 
attribute, or you may even track them in a list (maybe using weak references).

If sys.excepthook is used to handle threading exceptions, you call 
threading.current_thread(), but then we come back to the issue to "dying" 
Python: exception which occurs late in Python finalization, when most modules 
are already cleared and import no longer works.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 25/05/2019 à 23:09, STINNER Victor a écrit :
> 
> I don't see the relationship between the API (signature) and the ability to 
> set a per-thread hook.
> 
> I'm not convinced that a per-thread hook is needed.

Indeed, if you write your own Thread class, you can add a try...except
in the Thread.run() method.  You don't need a dedicated
Thread.excepthook() method.

The only way a per-thread hook could be useful is if you could set it
*outside* of the Thread class (so not as a method), so that one can e.g.
catch / report exceptions raised in threads launches by third-party
libraries.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Sorry, I had overlooked the issue with global variables at shutdown.  Though 
that issue only occurs with daemonic threads, since non-daemonic threads are 
joined before global variables are cleared.

In any case, I think the namedtuple / structseq solution is elegant, because we 
can add additional information later (the user must only be careful not to use 
tuple unpacking).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

Serhiy Storchaka:
> I propose to add the Thead.excepthook() method with the signature compatible 
> with sys.excepthook(). This will allow to set easily per-thread hooks and a 
> global hook.

I don't see the relationship between the API (signature) and the ability to set 
a per-thread hook.

I'm not convinced that a per-thread hook is needed. My proposed global 
threading.excepthook gets a thread parameter which allows a custom hook to 
implement a different behavior depending on the thread. You can use a different 
behavior depending on the thread name, depending on a custom Thread attribute, 
etc.

Would it be acceptable for first add a global hook and see later if a 
per-thread hook is needed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

> A Thread.excepthook() method does not allow to notify exceptions raised in 
> C-created threads ("dummy threads").

The main difference between sys.excepthook and threading.excepthook is that the 
threading hook displays the thread name. Which output do you expect if you 
don't pass a threading.Thread object (which has a 'name' attribute)? The thread 
identifier from _thread.get_ident()?

I can easily modify the default hook implementation to support args.thread=None 
and displays _thread.get_ident() as the name instead.


> Also, as I said already, you can get the current thread by calling 
> threading.current_thread() in the except hook. There is no need to pass it 
> explicitly to the except hook.

I understand that your plan is to use sys.excepthook to handle threading.Thread 
uncaught exception.

My main concern is that sys.excepthook can fail on importing the threading 
module. One solution would be to import threading when the sys module is 
created and keep a strong reference to threading.current_thread(). But I 
dislike this idea.

I already raised my concern, but so far, you didn't explain how you plan to fix 
this practical issue.

Moreover, the current threading code is quite complex. My guess is that this 
complexity is needed to display exception very late during Python shutdown, 
when module attributes are set to None and import no longer works.

I would prefer to not move this complexity into sys.excepthook which has 
currently a simple implementation.

--

Daemon threads are evil. We should drop this nasty feature... but I would 
prefer to not block this issue by the removal of daemon threads, since I'm sure 
that a lot of code rely on them and so it will be really hard to really remove 
them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Antoine Pitrou]

Antoine Pitrou  added the comment:

A Thread.excepthook() method does not allow to notify exceptions raised in 
C-created threads ("dummy threads").

Also, as I said already, you can get the current thread by calling 
threading.current_thread() in the except hook. There is no need to pass it 
explicitly to the except hook.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I propose to add the Thead.excepthook() method with the signature compatible 
with sys.excepthook(). This will allow to set easily per-thread hooks and a 
global hook.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

If you want to reuse sys.excepthook to handle uncaught Thread.run() exception, 
you can now write:
---
def hook(args):
if args.exc_type == SystemExit:
return
sys.excepthook(args.exc_type, args.exc_value, args.exc_traceback)

threading.excepthook = hook
---

Try attached sys_threading_excepthook.py for a full example.

--
Added file: https://bugs.python.org/file48354/sys_threading_excepthook.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

I rewrote my PR 13515:

* threading.excepthook() now gets a single argument which has multiple 
attributes: (exc_type, exc_value, exc_traceback, thread)

* The default threading.excepthook() implementation in written in C which 
reduces the risk of missing symbol during Python shutdown. There is also a 
simple implementation in Python, for other Python implementations which don't 
want to implement _thread._excepthook() in C.

* New _make_invoke_excepthook() function which handles the gory details for 
daemon threads. It creates a "local namespace" with references to all required 
functons and creates a _invoke_excepthook() function.

* _invoke_excepthook() packs arguments as a C structseq / Python namedtuple 
(depending on the implementation), calls threading.excepthook(). If 
threading.excepthook() raises an exception, sys.excepthook() is called to 
handle it.

--

First I also added a "stderr" argument to the arguments passed to 
threading.excepthook(): sys.stderr, or an old copy of sys.stderr if sys.stderr 
no longer exists or is set to None. But I decided to keep this as an 
implementation detail instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

There is a special case. If a thread calls os.fork() and Thread.run() raises an 
exception, the thread name is still logged even if there is only 1 thread after 
fork. Try attached fork_thread.py.

Output on Python 3.7:
---
main thread: spawn fork thread
thread: before fork [<_MainThread(MainThread, started 140623217481536)>, 
]
thread: after fork []
thread: after fork: raise
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
  File "fork_thread.py", line 17, in run
raise Exception("what happens here?")
Exception: what happens here?

main thread: done
---

Moreover, threading.Thread silently ignores SystemExit in run() even if it's 
the only remaining thread (main thread is gone after fork).

I don't think that we *have to* change the current behavior. It's just that we 
have to take it in account if we modify how exceptions are handled.

--
Added file: https://bugs.python.org/file48351/fork_thread.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Moreover, when sys.excepthook is called, the hook doesn't get access to the 
> thread object

You can get it with threading.current_thread(), no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Christoph Reiter]

Christoph Reiter  added the comment:

> Let's say that in Python 3.8 threading.Thread now calls sys.execpthook() to 
> handle uncaught run() exception. All applications which override 
> sys.excepthook() on purpose will behave differently: start to log exceptions 
> from threads. But existing code is unlikely to be prepared to implement black 
> magic to check if we are a "thread" or the main thread, to decide if we 
> should display a thread name, and also the "black magic" to get the current 
> thread name.

Note that PyErr_Print() and PyErr_PrintEx() can be called in threads, and 
CPython itself uses it in some places which can be called in threads and I also 
use it in thread callbacks in C extensions I work on (PyGObject and pycairo for 
example). Nothing states currently that it's not allowed to call it in such 
cases :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

About threading.excepthook() API, maybe we should not reproduce 
sys.excepthook() API but instead reuse something closer to sys.unraisablehook() 
API: use a single parameter which has attributes. It would allow to pass more 
parameters as new attributes in the future, maybe some new "optional" 
parameters (None by default).

For example, we can imagine calling threading.excepthook() to handle 
threading.excepthook() failure. We would need an argument to prevent an infine 
loop :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

I dislike PR 8610: threading.Thread doesn't call sys.excepthook to handle run() 
exception by default, it only calls sys.excepthook if it's overridden. 
Moreover, when sys.excepthook is called, the hook doesn't get access to the 
thread object :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

I wrote PR 13515 which adds threading.excepthook(). I chose to call 
threading.excepthook() even when run() raises SystemExit. In this case, 
threading.excepthook() simply does nothing. The idea is to really give the full 
control when threading.excepthook() is overriden. For example, log a warning 
when run() raises SystemExit. By the way, is it really a good idea to call 
sys.exit() from a thread? It sounds like a bug that should be reported, and not 
silently ignored, no?

Differences between sys.excepthook() and threading.excepthook():

* API: sys.excepthook(exctype, value, traceback, /) vs 
threading.excepthook(exc_type, exc_value, exc_tb, thread, /) -- addition thread 
parameter to display the name of the thread which raises an exception

* For SystemExit corner case, sys.excepthook() displays the exception, whereas 
threading.excepthook() silently ignores it

* When sys.stderr is None, sys.excepthook() does nothing, whereas 
threading.excepthook() tries harder: use its own copy of sys.stderr (saved when 
the thread has been created) from thread._stderr.

Thread._stderr was added by bpo-754449:

commit cc4e935ea593cede10cb1316e3faeabd708abca7
Author: Brett Cannon 
Date:   Sat Jul 3 03:52:35 2004 +

threading.Thread objects will now print a traceback for an exception raised
during interpreter shutdown instead of masking it with another traceback 
about
accessing a NoneType when trying to print the exception out in the first 
place.

Closes bug #754449 (using patch #954922).

Note: When sys.stderr is None, threading.excepthook() avoids the traceback 
module and renders the exception itself. Maybe threading.excepthook() should be 
reimplemented in C to make it even more reliable and more correct, especially 
during Python shutdown. Only daemon threads are impacted: Python finalization 
(Py_Finalize() C function) starts by calling threading._shutdown() which joins 
all non-daemon threads.


IMHO the threading.Thread semantics is too different than sys.excepthook() to 
reuse sys.excepthook() to handle threading.Thread.run() exception.

Another explanation is that sadly sys.excepthook() API uses exactly 3 
positional-only arguments, and so the API cannot be easily extended to get a 
thread argument. When I designed sys.unraisablehook(), I chose to pass only one 
argument which has attributes, to prevent this issue.

I'm not comfortable to attempt to modify sys.excepthook() to make it behave 
differently if it's called from the main thread or from a different thread. It 
would have to call threading.current_thread().name to get the name of the 
current thread.


Let's say that in Python 3.8 threading.Thread now calls sys.execpthook() to 
handle uncaught run() exception. All applications which override 
sys.excepthook() on purpose will behave differently: start to log exceptions 
from threads. But existing code is unlikely to be prepared to implement black 
magic to check if we are a "thread" or the main thread, to decide if we should 
display a thread name, and also the "black magic" to get the current thread 
name.


One of my concern of reusing sys.excepthook to display threading exceptions is 
that adding more code to handle threads is a risk of raising a new exception 
while logging a threading exception :-(

IMHO threading.excepthook() is safer since it already has access to the thread.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13432

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-36829: I just added sys.unraisablehook().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Christoph Reiter]

Christoph Reiter  added the comment:

To add one more use case for this:

concurrent.futures.Future.add_done_callback() currently ignores exceptions for 
the callback and just logs them which is easy to miss. I assume because it's 
not clear when and in which thread it gets called, and there is no "right" way 
to handle it.

It would be nice if it would call something like sys.excepthook instead.

--
nosy: +lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

My concern is more about backward compatibility. Documentation is one thing, 
but usually users rely on the actual implementation, not on the documentation, 
to define what is the Python behaviour.

Would you mind to open a thread on python-dev about this change?

I don't want to take the responsibility alone of such change :-) Moreover, I 
don't really have the bandwidth to work on this specific issue :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Andrey Vlasovskikh]

Andrey Vlasovskikh  added the comment:

> Would it be possible to modify the default implementation of sys.excepthook 
> to have a different output when it's not called from the main thread? Mimick 
> the current traceback from threads.

I agree it's a good idea to mimic what `_thread` does in case of both handled 
and unhandled exceptions. Why would you want to do it in `sys.excepthook`? The 
code for handling unhandled exceptions in `_thread` threads is located in 
`_threadmodule.c` itself, so there is no dependency from `sys` to `_thread`. By 
following the same logic, the code for handling unhandled exceptions in 
`threading` should be located in `threading.py`, like it does now. So the only 
thing that's missing in the pull request compared to `_thread` is the output:

   Unhandled exception in thread started by  at 
0x7f6769ca8e18>


for all the exceptions: not caught by the sys.excepthook **and** caught by the 
sys.exceptook.

> An alternative would be to introduce a new hook to log exceptions in threads, 
> ex: sys.threadexcepthook. That hoook would take a 4th parameter. But Andrey 
> Vlasovskikh didn't like this idea.

I'm overall OK with this idea, but I would prefer to make the existing 
`sys.excepthook` API applicable to all the exceptions: for the main thread 
(works well), for `_thread` threads (works well) and for `threading` threads 
(doesn't work).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Nikolaus Rath]

Change by Nikolaus Rath :


--
nosy:  -nikratio

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

Would it be possible to modify the default implementation of sys.excepthook to 
have a different output when it's not called from the main thread? Mimick the 
current traceback from threads.

Would it be possible to call threading.current_thread() from the default 
sys.excepthook? The problem is to get the threading module and to decide how to 
handle error on getting the current thread.

An alternative would be to introduce a new hook to log exceptions in threads, 
ex: sys.threadexcepthook. That hoook would take a 4th parameter. But Andrey 
Vlasovskikh didn't like this idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Antoine Pitrou]

Antoine Pitrou  added the comment:

No, I think this is a bug that deserves fixing, at least in 3.8.

--
versions: +Python 3.8 -Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Andrey Vlasovskikh]

Andrey Vlasovskikh  added the comment:

I've added a PR with a patch I developed during the EuroPython 2018 sprint.

I've fixed this issue in a way that is more or less consistent with how 
'_thread' threads interact with sys.excepthook, but I haven't added the log 
line they print to sys.stderr about an unhandled exception in a thread:

Unhandled exception in thread started by  at 0x108c840d0>

I can add these messages if they are needed.

I'd like to mention the issue I discussed with Victor Stinner at the sprints 
(adding Victor to the nosy list as he requested). The fix could possibly break 
the de facto contract that has been there for ages that 'theading' threads 
don't invoke sys.excepthook on uncaught exceptions.

If the fact that sys.execepthook doesn't work with threading is considered a 
feature, then an alternative solution could be a new threading-specific hook: 
threading.excepthook.

--
nosy: +vlasovskikh, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Andrey Vlasovskikh]

Change by Andrey Vlasovskikh :


--
keywords: +patch
pull_requests: +8116
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Nick Coghlan]

Change by Nick Coghlan :


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Antoine Pitrou]

Antoine Pitrou added the comment:

Matt, do you want to post your patch here?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Matt Groth]

Matt Groth added the comment:

Thank you Antoine Pitrou, I was able to disable this behavior by commenting out 
some lines of code in 'traceback' and replacing them with the appropriate call 
to 'sys.excepthook'. Note you also have to comment out a few lines in 
"Modules/_threadmodule.c" to correspond to this change or else that file 
complains that there wasn't the expected output printed to stderr.

--
nosy: +Matt Groth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Antoine Pitrou]

Antoine Pitrou added the comment:

Apparently the `traceback` module has been used from the start (since changset 
7f5013a9a9678e86bff00c97f98e7a92c515b94d) to print the exception and not 
sys.excepthook.  I presume this is an oversight, as I don't see any reason to 
prefer `traceback` here.  Tim, any insight?

--
nosy: +pitrou, tim.peters
stage: test needed -> needs patch
type:  -> behavior
versions: +Python 3.6, Python 3.7 -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[CyberJacob]

CyberJacob added the comment:

Just confirmed, this does affect multiprocessing. script to reproduce:

import sys, multiprocessing

def log_exception(*args):
print 'got exception %s' % (args,)

sys.excepthook = log_exception

def foo():
a = 1 / 0

multiprocessing.Process(target=foo).start()

again, just a normal traceback instead of "got exception"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Jacob Mansfield]

Jacob Mansfield added the comment:

Does this affect threads started with the multiprocessing library as well?

--
nosy: +Jacob Mansfield

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Decorater]

Decorater added the comment:

Ok, so I just found out you can bypass thread exceptions by wraping the line 
that actually runs the threads in a try/except block and then using the logging 
module to log it to a file instead of the console.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Decorater]

Decorater added the comment:

personally these exceptions in console can be annoying to me as I hate seeing 
them.

Exception in thread Thread-11:
Traceback (most recent call last):
  File "threading.py", line 914, in _bootstrap_inner
  File 
"E:\Users\Elsword\Desktop\DecoraterBot\Async\test\resources\Dependencies\discord\voice_client.py",
 line 150, in run
super().run()
  File 
"E:\Users\Elsword\Desktop\DecoraterBot\Async\test\resources\Dependencies\discord\voice_client.py",
 line 108, in run
self.player(data)
  File 
"E:\Users\Elsword\Desktop\DecoraterBot\Async\test\resources\Dependencies\discord\voice_client.py",
 line 669, in play_audio
sent = self.socket.sendto(packet, (self.endpoint_ip, self.voice_port))
OSError: [WinError 10055] An operation on a socket could not be performed 
because the system lacked sufficient buffer space or because a queue was full

Exception in thread Thread-21:
Traceback (most recent call last):
  File "threading.py", line 914, in _bootstrap_inner
  File 
"E:\Users\Elsword\Desktop\DecoraterBot\Async\test\resources\Dependencies\discord\voice_client.py",
 line 150, in run
super().run()
  File 
"E:\Users\Elsword\Desktop\DecoraterBot\Async\test\resources\Dependencies\discord\voice_client.py",
 line 108, in run
self.player(data)
  File 
"E:\Users\Elsword\Desktop\DecoraterBot\Async\test\resources\Dependencies\discord\voice_client.py",
 line 669, in play_audio
sent = self.socket.sendto(packet, (self.endpoint_ip, self.voice_port))
OSError: [WinError 10038] An operation was attempted on something that is not a 
socket

--
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Decorater]

Decorater added the comment:

I too agree that I hate the thread exceptions being printed in the console I 
would suggest python was to error log it all to a file instead (so it does not 
spam up the console). I get it a lot with websocket / opus errors and it is 
annoying because it does not cleanup ffmpeg for me.

--
nosy: +Decorater

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Nikolaus Rath]

Changes by Nikolaus Rath nikol...@rath.org:


--
nosy: +Nikratio

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1230540
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Terry J. Reedy]

Changes by Terry J. Reedy tjre...@udel.edu:


--
stage:  - unit test needed
versions: +Python 2.7, Python 3.2 -Python 2.5, Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1230540
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Ian Beaver]

Ian Beaver undercoverid...@gmail.com added the comment:

I found that the workaround suggested doesn't work when you have a
subclass of threading.Thread and you want to catch everything in the
module that contains the class to a common log.

Say you have a module with a socket server that spawns a thread on
accept and you want to log anything that tracebacks in the module. This
seems to work:

import sys
import logging
from functools import wraps

def myExceptHook(type, value, tb):
 Redirect tracebacks to error log 
import traceback
rawreport = traceback.format_exception(type, value, tb)
report = '\n'.join(rawreport)
log.error(report)

sys.excepthook = myExceptHook

def use_my_excepthook(view):
 Redirect any unexpected tracebacks  
@wraps(view)
def run(*args, **kwargs):
try:
return view(*args, **kwargs)
except:
sys.excepthook(*sys.exc_info())
return run


Then in your thread subclass:


class MyThread(threading.Thread):
def __init__(self, socket_conn):
threading.Thread.__init__(self)
self.my_conn = socket_conn

@use_my_excepthook
def run(self):
 Do stuff 

--
nosy: +undercoveridiot

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1230540
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Ian Beaver]

Ian Beaver undercoverid...@gmail.com added the comment:

Instead of using decorators, this is a slightly simpler modification to
the proposed workaround that allows for any subclassed run method to
also be caught.


def installThreadExcepthook():

Workaround for sys.excepthook thread bug
From
http://spyced.blogspot.com/2007/06/workaround-for-sysexcepthook-bug.html
   
(https://sourceforge.net/tracker/?func=detailatid=105470aid=1230540group_id=5470).
Call once from __main__ before creating any threads.
If using psyco, call psyco.cannotcompile(threading.Thread.run)
since this replaces a new-style class method.

init_old = threading.Thread.__init__
def init(self, *args, **kwargs):
init_old(self, *args, **kwargs)
run_old = self.run
def run_with_except_hook(*args, **kw):
try:
run_old(*args, **kw)
except (KeyboardInterrupt, SystemExit):
raise
except:
sys.excepthook(*sys.exc_info())
self.run = run_with_except_hook
threading.Thread.__init__ = init

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1230540
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Tiago Alves]

Tiago Alves added the comment:

I don't see it as a problem or as the threading module trying to be
clever. It clearly was a design choice to make the module have its own
exception treatment in order to make it clear in which thread the
exception occurred.
IMHO the decision here should be to implement per thread excepthook's.

--
nosy: +tiagoaoa

_
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1230540
_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[Christian Heimes]

Changes by Christian Heimes:


--
versions: +Python 2.5, Python 2.6 -Python 2.4

_
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1230540
_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com