[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +15183, 15184
pull_request: https://github.com/python/cpython/pull/15492

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +15183, 15184, 15185
pull_request: https://github.com/python/cpython/pull/15492

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +15183
pull_request: https://github.com/python/cpython/pull/15492

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests:  -15179

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Andrew Svetlov]

Change by Andrew Svetlov :


--
pull_requests: +15179
pull_request: https://github.com/python/cpython/pull/15492

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Yury Selivanov]

Yury Selivanov  added the comment:

I'll close this issue now.  If you find another bug in how asyncio handles 
async generators please open a new one. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset a14dda5df62369d01db6c7519e73aae71d0e7cfe by Yury Selivanov 
(twisteroid ambassador) in branch '3.6':
[3.6] bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716) 
(GH-9792)
https://github.com/python/cpython/commit/a14dda5df62369d01db6c7519e73aae71d0e7cfe


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[twisteroid ambassador]

Change by twisteroid ambassador :


--
pull_requests: +9174

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[miss-islington]

miss-islington  added the comment:


New changeset 41e5ec377b36aa951ac095839d2f74f66ee1e665 by Miss Islington (bot) 
in branch '3.7':
bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716)
https://github.com/python/cpython/commit/41e5ec377b36aa951ac095839d2f74f66ee1e665


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[miss-islington]

Change by miss-islington :


--
pull_requests: +9160

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[miss-islington]

Change by miss-islington :


--
pull_requests: +9161

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset c880ffe7d2ce2fedb1831918c8a36e3623e0fb76 by Yury Selivanov 
(twisteroid ambassador) in branch 'master':
bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716)
https://github.com/python/cpython/commit/c880ffe7d2ce2fedb1831918c8a36e3623e0fb76


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[twisteroid ambassador]

twisteroid ambassador  added the comment:

I’m now convinced that the bug we’re fixing and the original bug with debug 
mode off are two separate bugs. With the fix in place and debug mode off, I’m 
still seeing the original buggy behavior. Bummer.


In my actual program, I have an async generator that spawns two tasks. In the 
finally clause I cancel and wait for them, then check and log any exceptions. 
The program ran on Python 3.7. The symptom of the original bug is occasional 
“Task exception was never retrieved” log entries about one of the spawned 
tasks. After I patched 3.7 with the fix, the symptom remains, so the fix does 
not actually fix the original bug.

Running the same program on master, there are additional error and warning 
messages about open stream objects being garbage collected, unclosed sockets, 
etc. Are these logging messages new to 3.8? If not, perhaps 3.8 introduced 
additional bugs?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[twisteroid ambassador]

Change by twisteroid ambassador :


--
keywords: +patch
pull_requests: +9099
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[twisteroid ambassador]

twisteroid ambassador  added the comment:

I'll open a PR with your diff soon, but I don't have a reliable unit test yet. 
Also, it does not seem to fix the old problem with debug mode off. :-( I had 
hoped that the problem with debug mode off is nothing more than 
_asyncgen_finalizer_hook not running reliably each time, but that doesn't seem 
to be the case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Yury Selivanov]

Yury Selivanov  added the comment:

Rereading your first message:

> When testing my happy eyeballs library, I occasionally run into issues with 
> async generators seemingly not finalizing. After setting 
> loop.set_debug(True), I have been seeing log entries like these:

The bug we are fixing now is that async generators were not finalizing properly 
*in debug mode*.  

The "I occasionally run into issues with async generators seemingly not 
finalizing" part will need further investigation if you experienced it with 
debug mode off.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[Yury Selivanov]

Yury Selivanov  added the comment:

Wow.  Thanks so much for figuring this out, I know how hard it is to debug 
issues like this.

Now I see it clearly: _asyncgen_finalizer_hook should be using 
loop.call_soon_threadsafe.  Interestingly, I used _write_to_self there, so I 
knew about the issue, but figured that using call_soon() + _write_to_self is 
safe enough; evidently I was wrong.

Anyways, here's the diff that fixed it for me: 
https://gist.github.com/1st1/c1c9fc853cac1fadb7102ccc6201fb70

Could you please create a PR (ideally with a unittest)?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[twisteroid ambassador]

twisteroid ambassador  added the comment:

I have finally managed to reproduce this one reliably. The issue happens when 
i) async generators are not finalized immediately and must be garbage collected 
in the future, and ii) the garbage collector happens to run in a different 
thread than the one running the event loop. (Obviously, if there are more than 
one Python threads, eventually gc will run in those other threads, causing 
problems.)

I have attached a script reproducing the problem. I tried several ways of using 
async generators (the use_agen_*() coroutines), and the only way to make them 
not finalize immediately is use_agen_anext_separate_tasks(), which is the 
pattern used in my Happy Eyeballs library.

--
Added file: https://bugs.python.org/file47838/asyncgen_test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34769] _asyncgen_finalizer_hook running in wrong thread

[twisteroid ambassador]

New submission from twisteroid ambassador :

When testing my happy eyeballs library, I occasionally run into issues with 
async generators seemingly not finalizing. After setting loop.set_debug(True), 
I have been seeing log entries like these:


Exception ignored in: 
Traceback (most recent call last):
  File "/opt/Python3.7.0/lib/python3.7/asyncio/base_events.py", line 466, in 
_asyncgen_finalizer_hook
self.create_task(agen.aclose())
  File "/opt/Python3.7.0/lib/python3.7/asyncio/base_events.py", line 386, in 
create_task
task = tasks.Task(coro, loop=self)
  File "/opt/Python3.7.0/lib/python3.7/asyncio/base_events.py", line 674, in 
call_soon
self._check_thread()
  File "/opt/Python3.7.0/lib/python3.7/asyncio/base_events.py", line 712, in 
_check_thread
"Non-thread-safe operation invoked on an event loop other "
RuntimeError: Non-thread-safe operation invoked on an event loop other than the 
current one
ERRORasyncio Task was destroyed but it is pending!
source_traceback: Object created at (most recent call last):
  File "/opt/Python3.7.0/lib/python3.7/threading.py", line 885, in _bootstrap
self._bootstrap_inner()
  File "/opt/Python3.7.0/lib/python3.7/threading.py", line 917, in 
_bootstrap_inner
self.run()
  File "/opt/Python3.7.0/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
  File "/opt/Python3.7.0/lib/python3.7/concurrent/futures/thread.py", line 80, 
in _worker
work_item.run()
  File "/opt/Python3.7.0/lib/python3.7/concurrent/futures/thread.py", line 57, 
in run
result = self.fn(*self.args, **self.kwargs)
  File "/opt/Python3.7.0/lib/python3.7/asyncio/base_events.py", line 746, in 
_getaddrinfo_debug
msg.append(f'type={type!r}')
  File "/opt/Python3.7.0/lib/python3.7/enum.py", line 572, in __repr__
self.__class__.__name__, self._name_, self._value_)
  File "/opt/Python3.7.0/lib/python3.7/asyncio/base_events.py", line 466, in 
_asyncgen_finalizer_hook
self.create_task(agen.aclose())
  File "/opt/Python3.7.0/lib/python3.7/asyncio/base_events.py", line 386, in 
create_task
task = tasks.Task(coro, loop=self)
task: ()> created 
at /opt/Python3.7.0/lib/python3.7/asyncio/base_events.py:386>



This is a typical instance. Usually several instances like this occur at once.

I'll try to reproduce these errors in a simple program. Meanwhile, here are 
some details about the actual program, which may or may not be related to the 
errors:

* I have several nested async generators (async for item in asyncgen: yield 
do_something(item); ), and when the errors appear, the above error messages and 
stack traces repeat several times, with the object names mentioned in 
"Exception ignored in: ..." being each of the nested async generators. Other 
parts of the error messages, including the stack traces, are exactly the same.

* I never used threads or loop.run_in_executor() explicitly in the program. 
However, the innermost async generator calls loop.getaddrinfo(), and that is 
implemented by running a Python function, socket.getaddrinfo(), with 
loop.run_in_executor(), so the program implicitly uses threads. 
(socket.getaddrinfo() is a Python function that calls a C function, 
_socket.getaddrinfo().)

* The outermost async generator is not iterated using `async for`. Instead, it 
is iterated by calling its `__aiter__` method, saving the returned async 
iterator object, and then awaiting on the `__anext__` method of the async 
iterator repeatedly. Of course, all of these are done in the same event loop.


Environment: Python 3.7.0 compiled from source, on Debian stretch.

--
components: asyncio
messages: 326090
nosy: asvetlov, twisteroid ambassador, yselivanov
priority: normal
severity: normal
status: open
title: _asyncgen_finalizer_hook running in wrong thread
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com