[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Jack O'Connor]

New submission from Jack O'Connor:

The following hangs at 100% CPU on Python 3.5, though not on Python 3.4:

1) Start an asyncio coroutine with run_until_complete().
2) Inside the coroutine, enter an ExitStack using a with-statement.
3) Inside the with-statement, call ExitStack.enter_context() with a generator 
context manager. It doesn't matter what the generator yields.
4) After the enter_context() call, raise an exception.

Here's an example script that does all of this and repros the hang: 
https://gist.github.com/oconnor663/483db2820bb5f877c9ed

--
components: asyncio
messages: 255719
nosy: gvanrossum, haypo, oconnor663, yselivanov
priority: normal
severity: normal
status: open
title: deadlock with asyncio+contextmanager+ExitStack
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

Trying to reproduce without contextstack.

--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Guido van Rossum]

Guido van Rossum added the comment:

Interestingly, it doesn't hang when you raise a different error. There's some 
new code dealing with the RuntimeError coming out of a generator if it raises 
StopIteration (instead of returning) introduced by issue #22906. Yury, it looks 
like you introduced that? (The diff is this:

changeset:   95932:36a8d935c322
user:Yury Selivanov 
date:Sat May 09 11:44:30 2015 -0400
summary: PEP 479: Change StopIteration handling inside generators.

--
assignee:  -> yselivanov
keywords: +3.5regression

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

Here's a minimal test to reproduce:

import reprlib


def main():
if 0:
yield
raise RuntimeError


m = main()
try:
m.send(None)
except RuntimeError as ex:
ex.__context__ = ex
reprlib.repr(ex)


Looks like it's a bug in reprlib.  It's not related to PEP 492/479.  

It's also reproducible in Python 3.4 and 3.3.

Nick, ExitStack does this (indirectly) 'ex.__context__ = ex' thing -- I think 
that's a bug of contextlib.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Jack O'Connor]

Jack O'Connor added the comment:

Thanks for chasing this down. Yury, can you suggest a workaround?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

Created another issue for the reprlib bug: issue 25781.  It appears we don't 
even need a generator:

import reprlib

try:
raise RuntimeError
except RuntimeError as ex:
ex.__context__ = ex
reprlib.repr(ex)

Closing this one with "not a bug".

--
resolution:  -> not a bug
status: open -> closed
superseder:  -> infinite loop in reprlib

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Changes by Yury Selivanov :


--
superseder: infinite loop in reprlib -> CPython hangs on error __context__ set 
to the error itself

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

It's not even a reprlib bug:

try:
raise Exception
except Exception as ex:
ex.__context__ = ex
hasattr(1, 'aa')

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

> Thanks for chasing this down. Yury, can you suggest a workaround?

I'm not sure how to workaround this :(  Hopefully we can fix this in 3.5.1.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

FWIW the bug was identified in issue 25782.  I've drafted a patch to fix it, 
please review.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

The question is whether we should raise an exception or not:

   ex.__context__ = ex

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

Another issue for contextlib: http://bugs.python.org/issue25786

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Jack O'Connor]

Jack O'Connor added the comment:

Yury, can you help me understand why `hasattr("foo", "bar")` triggers the 
infinite loop there, but not `print("foo")`?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25779] deadlock with asyncio+contextmanager+ExitStack

[Yury Selivanov]

Yury Selivanov added the comment:

> Yury, can you help me understand why `hasattr("foo", "bar")` triggers the 
> infinite loop there, but not `print("foo")`?


hasattr uses getattr under the hood. getattr raises an AttributeError, and that 
triggers PyErr_SetError, which has an infinite "while" loop.  Instead of 
"hasattr" you can use anything that raises an error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com