[issue34679] asyncio.add_signal_handler call fails if not on main thread

[miss-islington]

miss-islington  added the comment:


New changeset cbf474c98e702d12c97cd16a1e44ede10ea52b5b by Miss Skeleton (bot) 
in branch '3.8':
bpo-34679: ProactorEventLoop only uses set_wakeup_fd() in main thread (GH-16901)
https://github.com/python/cpython/commit/cbf474c98e702d12c97cd16a1e44ede10ea52b5b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[miss-islington]

Change by miss-islington :


--
pull_requests: +16439
pull_request: https://github.com/python/cpython/pull/16902

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 1b53a24fb4417c764dd5933bce505f5c94249ca6 by Victor Stinner in 
branch 'master':
bpo-34679: ProactorEventLoop only uses set_wakeup_fd() in main thread (GH-16901)
https://github.com/python/cpython/commit/1b53a24fb4417c764dd5933bce505f5c94249ca6


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +16437
pull_request: https://github.com/python/cpython/pull/16901

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[miss-islington]

miss-islington  added the comment:


New changeset 69d22b8fee442c12829a1032a72489c8133de271 by Miss Islington (bot) 
in branch '3.8':
bpo-34679: Restore instantiation Windows IOCP event loop from non-main thread 
(GH-15492)
https://github.com/python/cpython/commit/69d22b8fee442c12829a1032a72489c8133de271


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[miss-islington]

Change by miss-islington :


--
pull_requests: +15198
pull_request: https://github.com/python/cpython/pull/15512

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 1c0600998681295735a18690fae184b0c9a4ca51 by Andrew Svetlov in 
branch 'master':
bpo-34679: Restore instantiation Windows IOCP event loop from non-main thread 
(#15492)
https://github.com/python/cpython/commit/1c0600998681295735a18690fae184b0c9a4ca51


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Kyle Stanley]

Kyle Stanley  added the comment:

> Kyle, thanks for the fix.
> I have basically the same change in my PR but with test and news note.

No problem, that works for me. I was mostly just trying to help with resolving 
some of the release blockers for 3.8b4.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
keywords: +patch
pull_requests: +15180
pull_request: https://github.com/python/cpython/pull/15492

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Kyle, thanks for the fix.
I have basically the same change in my PR but with test and news note.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The issue is related to Python 3.8 and master only. 3.6-3.7 are not affected

--
versions: +Python 3.9 -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Kyle Stanley]

Change by Kyle Stanley :


--
keywords: +needs review -patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Kyle Stanley]

Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +15163
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15477

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Kyle Stanley]

Kyle Stanley  added the comment:

> How do we identify whether or not set_wakeup_fd() is being called from a 
> non-main thread?

Never mind, I think I found the answer to my own question and tested a patch 
locally, I'll open a PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Kyle Stanley]

Kyle Stanley  added the comment:

> Skipping this call for non-main thread in proactor implementation makes sense.

How do we identify whether or not set_wakeup_fd() is being called from a 
non-main thread?

--
nosy: +aeros167

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Brett Slatkin]

Brett Slatkin  added the comment:

Maybe we should just roll back https://bugs.python.org/issue34687 to fix both 
issues? Otherwise asyncio will be broken on Windows in 3.8. Is general API 
stability more important than performance?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Yury Selivanov]

Yury Selivanov  added the comment:

Andrew, can you fix ctrl-c/windows issue? I'm basically afk until monday. 

And we're not going to do anything about add_signal_handler in 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Łukasz Langa]

Łukasz Langa  added the comment:

Please fix this ASAP, last 3.8 beta is scheduled for Monday.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Łukasz Langa]

Change by Łukasz Langa :


--
nosy: +lukasz.langa, ned.deily
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Good point.
For proactor event loop set_wakeup_fd() is used for Ctrl+C handling.
Skipping this call for non-main thread in proactor implementation makes sense.

--
assignee:  -> asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Brett Slatkin]

Brett Slatkin  added the comment:

I believe the scope of this bug may be larger than it originally seemed.

Now that ProactorEventLoop is the default for Python 3.8 
(https://bugs.python.org/issue34687), I'm seeing this same problem on Windows 
when you try to call asyncio.new_event_loop() from within a thread. It breaks 
with the ProactorEventLoop (snippet #1 below). It works fine with the 
SelectorEventLoop (snippet #2 below).

Am I wrong to expect to be able to create a unique event loop for each thread 
from within the thread itself?

I worked around this problem by manually forcing the event loop policy 
(https://bugs.python.org/issue33792).

=== Snippet #1 (breaks) ===

import asyncio
from threading import Thread

def my_func():
asyncio.new_event_loop()

t = Thread(target=my_func)
t.start()
t.join()

=== Output from snippet #1 ===

PS G:\> python .\repro.py
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\threading.py", 
line 932, in _bootstrap_inner
self.run()
  File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\threading.py", 
line 870, in run
self._target(*self._args, **self._kwargs)
  File ".\repro.py", line 6, in my_func
asyncio.new_event_loop()
  File 
"C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\events.py", 
line 758, in new_event_loop
return get_event_loop_policy().new_event_loop()
  File 
"C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\events.py", 
line 656, in new_event_loop
return self._loop_factory()
  File 
"C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\windows_events.py",
 line 310, in __init__
super().__init__(proactor)
  File 
"C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\proactor_events.py",
 line 630, in __init__
signal.set_wakeup_fd(self_no)
ValueError: set_wakeup_fd only works in main thread

=== Snippet #2 (works) ===

import asyncio
from threading import Thread

# Work-around from https://bugs.python.org/issue34679
policy = asyncio.get_event_loop_policy()
policy._loop_factory = asyncio.SelectorEventLoop

def my_func():
asyncio.new_event_loop()

t = Thread(target=my_func)
t.start()
t.join()

=== More details ===

My version of Python:

3.8.0b2 (tags/v3.8.0b2:21dd01d, Jul  4 2019, 16:00:09) [MSC v.1916 64 bit 
(AMD64)]

My version of Windows (it's a developer VM from 
https://developer.microsoft.com/en-us/windows/downloads/virtual-machines):

PS G:\> [System.Environment]::OSVersion.Version

Major  Minor  Build  Revision
-  -  -  
10 0  17763  0

--
nosy: +Brett Slatkin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Andrew Svetlov]

Andrew Svetlov  added the comment:

In my opinion, we should don't change the behavior.
Signal handlers should be processed in main thread at the end, that's how 
Python works for regular synchronous API.

Mentioning it in the doc makes sense.

Would you make a pull request?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[epiphyte]

Change by epiphyte :


--
nosy: +epiphyte

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34679] asyncio.add_signal_handler call fails if not on main thread

[Nic Watson]

New submission from Nic Watson :

Summary:  essentially asyncio.add_signal_handler doesn't work when called off 
the main thread.  One might consider this a documentation failure.

(Note: there's also a bigger issue with cleanup, which I'll submit separately)

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/asyncio/unix_events.py", 
line 91, in add_signal_handler
signal.set_wakeup_fd(self._csock.fileno())
ValueError: set_wakeup_fd only works in main thread

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/threading.py", line 916, 
in _bootstrap_inner
self.run()
  File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/threading.py", line 864, 
in run
self._target(*self._args, **self._kwargs)
  File "/home/nic/tmp/signal_event_loop_bug.py", line 14, in do_loop
loop.add_signal_handler(signal.SIGINT, mysighandler)
  File "/home/nic/.pyenv/versions/3.6.4/lib/python3.6/asyncio/unix_events.py", 
line 93, in add_signal_handler
raise RuntimeError(str(exc))
RuntimeError: set_wakeup_fd only works in main thread

Code:

import asyncio
import signal
import threading

def mysighandler():
pass

def do_loop():
loop = asyncio.new_event_loop()
# This will fail with RuntimeError: set_wakeup_fd only works in main thread
loop.add_signal_handler(signal.SIGINT, mysighandler)
loop.close()

t = threading.Thread(target=do_loop)
t.start()
t.join()

--
components: asyncio
messages: 325350
nosy: asvetlov, jnwatson, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.add_signal_handler call fails if not on main thread
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com