Re: [Async-sig] awaiting task is not chaining exception

2017-11-12 Thread Nathaniel Smith 
On Sun, Nov 12, 2017 at 7:27 AM, Guido van Rossum  wrote:
> On Sun, Nov 12, 2017 at 2:53 AM, Chris Jerdonek 
> wrote:
>>
>> By the way, since we're already on the subject of asyncio tasks and
>> (truncated) stack traces, this looks like a good opportunity to ask a
>> question that's been on my mind for a while:
>>
>> There's a mysterious note at the end of the documentation of
>> asyncio.Task's get_stack() method, where it says--
>>
>> > For reasons beyond our control, only one stack frame is returned for a
>> > suspended coroutine.
>>
>> (https://docs.python.org/3/library/asyncio-task.html#asyncio.Task.get_stack
>> )
>>
>> What does the "For reasons beyond our control" mean? What is it that
>> can possibly be beyond the control of Python?
>
>
> It's an odd phrasing, but it refers to the fact that a suspended generator
> frame (which is what a coroutine really is) does not have a "back link" to
> the frame that called it. Whenever a generator yields (or a coroutine
> awaits) its frame is disconnected from the current call stack, control is
> passed to the top frame left on that stack, and the single generator frame
> is just held on to by the generator object. When you call next() on that, it
> will be pushed on top of whatever is the current stack (i.e. whatever calls
> next()), which *may* be a completely different stack configuration than when
> it was suspended.

It is possible to get the await/yield from stack though, even when a
generator/coroutine is suspended, using the gi_yieldfrom / cr_await
attributes. Teaching Task.get_stack to do this would be a nice little
enhancement.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] awaiting task is not chaining exception

2017-11-12 Thread Guido van Rossum 
On Sun, Nov 12, 2017 at 2:53 AM, Chris Jerdonek 
wrote:

> By the way, since we're already on the subject of asyncio tasks and
> (truncated) stack traces, this looks like a good opportunity to ask a
> question that's been on my mind for a while:
>
> There's a mysterious note at the end of the documentation of
> asyncio.Task's get_stack() method, where it says--
>
> > For reasons beyond our control, only one stack frame is returned for a
> suspended coroutine.
> (https://docs.python.org/3/library/asyncio-task.html#
> asyncio.Task.get_stack )
>
> What does the "For reasons beyond our control" mean? What is it that
> can possibly be beyond the control of Python?
>

It's an odd phrasing, but it refers to the fact that a suspended generator
frame (which is what a coroutine really is) does not have a "back link" to
the frame that called it. Whenever a generator yields (or a coroutine
awaits) its frame is disconnected from the current call stack, control is
passed to the top frame left on that stack, and the single generator frame
is just held on to by the generator object. When you call next() on that,
it will be pushed on top of whatever is the current stack (i.e. whatever
calls next()), which *may* be a completely different stack configuration
than when it was suspended.

That's all.

-- 
--Guido van Rossum (python.org/~guido)
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] awaiting task is not chaining exception

2017-11-12 Thread Chris Jerdonek 
By the way, since we're already on the subject of asyncio tasks and
(truncated) stack traces, this looks like a good opportunity to ask a
question that's been on my mind for a while:

There's a mysterious note at the end of the documentation of
asyncio.Task's get_stack() method, where it says--

> For reasons beyond our control, only one stack frame is returned for a 
> suspended coroutine.
(https://docs.python.org/3/library/asyncio-task.html#asyncio.Task.get_stack )

What does the "For reasons beyond our control" mean? What is it that
can possibly be beyond the control of Python?

--Chris


On Sat, Nov 11, 2017 at 1:47 AM, Chris Jerdonek
 wrote:
> On Sat, Nov 11, 2017 at 12:39 AM, Nathaniel Smith  wrote:
>> On Fri, Nov 10, 2017 at 9:52 PM, Chris Jerdonek
>>  wrote:
>>> Hi, I recently encountered a situation with asyncio where the stack
>>> trace is getting truncated: an exception isn't getting chained as
>>> expected.
>>>
>>> I was able to reduce it down to the code below.
>>>
>>> The reduced case seems like a pattern that can come up a lot, and I
>>> wasn't able to find an issue on the CPython tracker, so I'm wondering
>>> if I'm doing something wrong or if the behavior is deliberate.
>>
>> I think what you're seeing is collateral damage from some known
>> bugginess in the generator/coroutine .throw() method:
>> https://bugs.python.org/issue29587
>
> Ah, thanks for the great explanation, Nathaniel!
>
> From the original bug report Nathaniel filed above:
>> It's likely that more people will run into this in the future as async/await 
>> becomes more widely used. ...
>
> :)
>
> --Chris
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] awaiting task is not chaining exception

2017-11-11 Thread Nathaniel Smith 
On Fri, Nov 10, 2017 at 9:52 PM, Chris Jerdonek
 wrote:
> Hi, I recently encountered a situation with asyncio where the stack
> trace is getting truncated: an exception isn't getting chained as
> expected.
>
> I was able to reduce it down to the code below.
>
> The reduced case seems like a pattern that can come up a lot, and I
> wasn't able to find an issue on the CPython tracker, so I'm wondering
> if I'm doing something wrong or if the behavior is deliberate.

I think what you're seeing is collateral damage from some known
bugginess in the generator/coroutine .throw() method:
https://bugs.python.org/issue29587

In trio I work around this by never using throw(); instead I send() in
the exception and re-raise it inside the coroutine:
https://github.com/python-trio/trio/blob/389f1e1e01b410756e2833cffb992fd1ff856ae5/trio/_core/_run.py#L1491-L1498

But asyncio doesn't do this -- when an asyncio.Task awaits an
asyncio.Future and the Future raises, the exception is throw()n into
the Task, triggering the bug:
https://github.com/python/cpython/blob/e184cfd7bf8bcfd160e3b611d4351ca3ce52d9e2/Lib/asyncio/tasks.py#L178

(If you try profiling your code you may also see weird/impossible
results in cases like this, because throw() also messes up stack
introspection.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/