[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

> Without thread support, event generation from multiple threads fails 
> immediately.

This ticket is for threaded Tcl only, so this is off topic.

In nonthreaded Tcl, this script crashes rather than freezes, for an entire ly 
different reason that I already explained in https://bugs.python.org/issue33257 
.

This ticket is solved if you ask me.
The only remaining matter is that there's no documentation:

* on Tkinter threading model: https://docs.python.org/3/library/tk.html claims 
full thread safety which is untrue.
* on best practices with Tkinter: as you could see, all the more or less 
obvious solutions are flawed. (That includes my solution: the program doesn't 
terminate gracefully if you close the window by hand.)

I'm going to cover at least the first item as part of executing Guido's 
suggestion to "add a warning to the docs".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Without thread support, event generation from multiple threads fails 
immediately.  I tried an experiment with callback scheduling.  It seems to work 
-- almost.

thread_event.py runs on 2.7 with non-t tcl.  It modifies TkinterHandlres32.py 
by replacing
self.target.event_generate(c)
with
self.target.after(1, lambda t=self.target: t.event_generate(c))
to schedule the event generation in the main thread.
It also imports either tkinter or Tkinter, and runs for 10 seconds
self.root.after(1,self.stop)
for a more rigorous test.

However, when I add 2 0s to the delay, to make it 1000 seconds, the main thread 
and gui crash sometime sooner (100 seconds, say), leaving the worker threads 
sending indefinitely.  One time there was a traceback:

Traceback (most recent call last):
  File "F:\dev\tem\thread_event.py", line 55, in 
Main().go()
  File "F:\dev\tem\thread_event.py", line 35, in go
self.t_cleanup.join()
AttributeError: 'Main' object has no attribute 't_cleanup'

A second time, nothing appeared.

I suspect that without proper locking an .after call was eventually interrupted 
and the pending scheduled callback data structure corrupted. Mainloop exits 
without t_cleanup created.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Terry J. Reedy]

Terry J. Reedy  added the comment:

When I closed the main thread by clicking [x], thus destroying root, both event 
threads raised instead of hanging.  So my experiment involved calling 
root.destroy instead of setting running to False.  The better result when 
running under IDLE might be due to IDLE's run process executes 
tcl.call('update') about 20 times per second.  Even if the exception idea could 
be made to work, it seems like a kludge.

Waiting on the event threads from a separate non-gui thread and leaving the 
main thread fully functional and responsive until the gui threads die seems 
much cleaner.  Perhaps this should be recommended as a standard way to shut 
down the main thread when there might be active gui threads.  Thank you for 
following through with this to get a solution we both like.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

Attached a fixed script.

`Tk.after()` works from a worker thread, while `Tk.destroy()` doesn't.

That's because Tkinter implements Tcl calls (_tkinter.c:Tkapp_Call) from 
another thread by posting an event to the interpreter's queue 
(Tcl_ThreadQueueEvent) and waiting for result. So a call normally works, but 
would hang if the interpreter's event loop is not running.

`destroy()`'s Python part (Lib\tkinter\__init__.py:2055) stops the event loop, 
then makes more Tcl calls -- which hang for the aforementioned reason if made 
from another thread.

--
Added file: https://bugs.python.org/file47570/TkinterHanders32.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

> Another possibility is for stop() to change conditions so that 
> 'self.target.event_generate(c)' fails with an exception

Could you elaborate? Since there're no docs on event_generate(), I can't look 
up how to make it "fail with an exception" without actually posting an event.


> The only problem is that the first t.join() hangs because of a thread 
> deadlock bug.  t.join() blocks until t.run exits.  t.run does not exit until 
> the last event_generate, with running=False, returns.  But that blocks until 
> dummy_handler runs.
> I suppose there is a teeny possibility that 'running' could be flipped 
> between the test and the call.  Can that be prevented with a lock?

The idea is to let the worker threads finish their work, not terminate them 
forcibly.
So the real problem is that stop() blocks the event loop.
It should rather run asynchronously, wait for threads, then trigger 
`self.root.destroy()` in the main thread... somehow.

--
resolution: wont fix -> not a bug

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Terry J. Reedy]

Terry J. Reedy  added the comment:

>From 1994 to 2017, _tkinter.c has received a steady flow of multiple revisions 
>each year, for 333 total, by maybe 20 people including Guido.  This makes it 
>one of the more actively maintained files and indicates the opposite of 
>indifference and not caring.

I tested event generation from threads on my Win 10 machine.  With installed 
2.7, with non-thread tcl 8.5, the process hangs as described, after 1 key event 
is sent and received.  So we should document "don't do that".

With installed 64-bit 3.6 with thread-compiled tcl 8.6, I see something 
completely different.  The process runs for 5 seconds until the stop call.  The 
two threads alternate sending key events, which are all received in the main 
thread.  Ditto for built 32-big debug 3.6 and 3.8.  

The only problem is that the first t.join() hangs because of a thread deadlock 
bug.  t.join() blocks until t.run exits.  t.run does not exit until the last 
event_generate, with running=False, returns.  But that blocks until 
dummy_handler runs.  Dummy_handler does not run when the main thread is blocked 
by t.join.

I fixed this by not generating events when running is False.  The revised 
program exits properly.

def run(self):
tid = self.ident
while True:
time.sleep(0.02)
c = random.choice(string.ascii_letters)
print(running, "%d: sending '%s'"%(tid,c),file=sys.stderr)
if running:
self.target.event_generate(c)
else:
break

I suppose there is a teeny possibility that 'running' could be flipped between 
the test and the call.  Can that be prevented with a lock?

Another possibility is for stop() to change conditions so that 
'self.target.event_generate(c)' fails with an exception, and change if/else to 
for/except.  My first try sort of works in IDLE but not the console.

--
nosy: +serhiy.storchaka, terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Guido van Rossum]

Guido van Rossum  added the comment:

I guess nobody gives a damn.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

> 
Do you have a suggestion for what to do short of dropping Tkinter support?

Didn't really look into this.
At first glance, from the trace log, the main thread seems to grab a lock at 
some initial point, and then tries to grab it again when running an event 
handler. So making the lock reentrant may help.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Guido van Rossum]

Guido van Rossum  added the comment:

So it seems threads and Tkinter events don't mix. This doesn't surprise me 
much. (Similar issues can occur when mixing threads and asyncio if you don't 
follow the documentation's advice about how to send events across threads.)

Perhaps event_generate() needs to be more careful with locking?

Do you have a suggestion for what to do short of dropping Tkinter support?

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

> worker threads are waiting for the Tcl lock

Pardon. They are waiting for Tkapp_ThreadSend()s into the main thread to 
return. The effect is still the same.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

Change by Ivan Pozdeev :


Added file: https://bugs.python.org/file47564/trace.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

New submission from Ivan Pozdeev :

With threaded Tkinter, TkinterHanders3.py from 
https://bugs.python.org/issue33257 (attached) hangs.

Tracing with thread_debug and a modified trace.py (to show TIDs, attached) 
shows that worker threads are waiting for the Tcl lock while the main thread 
that holds it keeps waiting for some other lock with a strange timeout:

19000: PyThread_acquire_lock_timed(001B0F80, 0) called
19000: PyThread_acquire_lock(001B0F80, 0) -> 0
19000: PyThread_acquire_lock_timed(001B0F80, -100) called

Tested on 3.6 head, win7 x64, debug build.

--
components: Tkinter
files: TkinterHanders3.py
messages: 316082
nosy: Ivan.Pozdeev
priority: normal
severity: normal
status: open
title: Tkinter hangs if using multiple threads and event handlers
type: crash
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file47562/TkinterHanders3.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33412] Tkinter hangs if using multiple threads and event handlers

[Ivan Pozdeev]

Change by Ivan Pozdeev :


Added file: https://bugs.python.org/file47563/trace.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com