Re: [Async-sig] [ANN] async_generator v1.2 released

[Nathaniel Smith]

On Fri, Nov 25, 2016 at 10:46 AM, Alex Grönholm
 wrote:
> 25.11.2016, 12:09, Nathaniel Smith kirjoitti:
>>
>> On Thu, Nov 24, 2016 at 11:59 PM, Alex Grönholm
>>  wrote:
>>>
>>> 25.11.2016, 09:25, Nathaniel Smith kirjoitti:

 On Thu, Nov 24, 2016 at 1:23 PM, Nathaniel Smith  wrote:
 [...]
>>
>> One thing I noticed is that there seems to be no way to detect async
>> generator functions in your implementation. That is something I would
>> want
>> to have before switching.
>
> Good point. That should be pretty trivial to add.

 Just pushed async_generator v1.3 to PyPI, with new isasyncgen,
 isasyncgenfunction, and (on 3.6) registration with
 collections.abc.AsyncGenerator.
>>>
>>> And you just *had* to make it incompatible with the async generators from
>>> asyncio_extras? "_is_async_gen_function" vs "_is_async_generator"?
>>> I thought we agreed on cooperating?
>>
>> I started doing that, but... it's not an async generator, isasyncgen
>> is a different inspect function...
>>
>> -n
>>
> It's an arbitrary string that will likely never be seen by anyone except for
> people working on the libraries. But it makes a world of difference in
> compatibility. I named it this way to be consistent with asyncio which marks
> yield-based coroutines with "_is_coroutine". So please reconsider.

Sure, I guess it doesn't matter too much either way.

Can we pause a moment to ask whether we really want the two async
generator types to return true for each other's introspection
function, though? Right now they aren't really interchangeable, and
the only user for this kind of introspection I've seen is your
contextmanager.py. It seems to use the inspect versions of isasyncgen
and isasyncgenfunction, because it wants to know whether
asend/athrow/aclose are supported. Right now, if it switched to using
async_generator's isasyncgen/isasyncgenfunction, it would continue to
work; but if I then switched async_generator's
isasyncgen/isasyncgenfunction to detect asyncio_extras's generators,
it would stop working again.

Speaking of which, what do you want to do about isasyncgen?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] [ANN] async_generator v1.2 released

[Nathaniel Smith]

On Fri, Nov 25, 2016 at 12:03 AM, Alex Grönholm
 wrote:
> 24.11.2016, 23:23, Nathaniel Smith kirjoitti:
>
> On Nov 23, 2016 11:29 PM, "Alex Grönholm"  wrote:
>>
>> 23.11.2016, 01:34, Nathaniel Smith kirjoitti:
>>>
>>> On Tue, Nov 22, 2016 at 2:22 PM, Alex Grönholm 
>>> wrote:
>>> > I'm not sure where asyncio_extras's async generator implementation
>>> > assumes
>>> > you're using asyncio. Could you elaborate on that?
>>>
>>> If I'm reading it right, it assumes that the only two things that might
>>> be yielded to the coroutine runner are either (a) the special yield wrapper,
>>> or (b) an awaitable object like an asyncio.Future. This works on asyncio,
>>> because that's all the asyncio runner supports, but it doesn't work with
>>> (for example) curio. async_generator (like native async generators) allows
>>> arbitrary objects to be yielded to the coroutine runner.
>>
>> You are misreading the code. It is in no way limited to what asyncio
>> accepts. It doesn't even import asyncio in the asyncyield or generator
>> modules. The only parts of the library that depend on PEP 3156 event loops
>> are the ones that involve executors and threads.
>
> I didn't say that it imported asyncio. I said that it assumes the only
> things that will be yielded are the things that asyncio yields. This is the
> line that I'm worried about:
>
>
> https://github.com/agronholm/asyncio_extras/blob/aec412e1b7034ca3cad386c381e655ce3547fee3/asyncio_extras/asyncyield.py#L40
>
> The code awaits the value yielded by the coroutine, but there's no guarantee
> that this value is awaitable. It's an arbitrary Python object representing a
> message sent to the coroutine runner. It turns out that asyncio only uses
> awaitable objects for its messages, so this code can get away with this on
> asyncio, but if you try using this code with curio then I'm pretty sure
> you're going to end up doing something like "await (3,)" and then blowing
> up.
>
> PEP 492 clearly states the following:
>
> It is a TypeError to pass anything other than an awaitable object to an
> await expression.
>
> That (3,) is not an awaitable, so the example is invalid. That said, I will
> re-examine this part of the implementation and correct it if necessary.

I feel like I'm running out of ideas for how to explain this, and
starting to just repeat myself :-(. There is nothing that says the
return value from coro.__next__() must be awaitable. Coroutines can
yield arbitrary objects. (3,) is not awaitable, but it's a perfectly
valid thing to be yielded from a coroutine.

> So far I just haven't encountered anything that would produce an error.

That's because you've only tested on asyncio, not curio. I promise!

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] [ANN] async_generator v1.2 released

[Alex Grönholm]

25.11.2016, 09:25, Nathaniel Smith kirjoitti:

On Thu, Nov 24, 2016 at 1:23 PM, Nathaniel Smith  wrote:
[...]

One thing I noticed is that there seems to be no way to detect async
generator functions in your implementation. That is something I would want
to have before switching.

Good point. That should be pretty trivial to add.

Just pushed async_generator v1.3 to PyPI, with new isasyncgen,
isasyncgenfunction, and (on 3.6) registration with
collections.abc.AsyncGenerator.
And you just *had* to make it incompatible with the async generators 
from asyncio_extras? "_is_async_gen_function" vs "_is_async_generator"?

I thought we agreed on cooperating?


-n



___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/

Re: [Async-sig] [ANN] async_generator v1.2 released

[Alex Grönholm]

24.11.2016, 23:23, Nathaniel Smith kirjoitti:


On Nov 23, 2016 11:29 PM, "Alex Grönholm" > wrote:

>
> 23.11.2016, 01:34, Nathaniel Smith kirjoitti:
>>
>> On Tue, Nov 22, 2016 at 2:22 PM, Alex Grönholm 
> wrote:
>> > I'm not sure where asyncio_extras's async generator 
implementation assumes

>> > you're using asyncio. Could you elaborate on that?
>>
>> If I'm reading it right, it assumes that the only two things that 
might be yielded to the coroutine runner are either (a) the special 
yield wrapper, or (b) an awaitable object like an asyncio.Future. This 
works on asyncio, because that's all the asyncio runner supports, but 
it doesn't work with (for example) curio. async_generator (like native 
async generators) allows arbitrary objects to be yielded to the 
coroutine runner.

>
> You are misreading the code. It is in no way limited to what asyncio 
accepts. It doesn't even import asyncio in the asyncyield or generator 
modules. The only parts of the library that depend on PEP 3156 event 
loops are the ones that involve executors and threads.


I didn't say that it imported asyncio. I said that it assumes the only 
things that will be yielded are the things that asyncio yields. This 
is the line that I'm worried about:


https://github.com/agronholm/asyncio_extras/blob/aec412e1b7034ca3cad386c381e655ce3547fee3/asyncio_extras/asyncyield.py#L40

The code awaits the value yielded by the coroutine, but there's no 
guarantee that this value is awaitable. It's an arbitrary Python 
object representing a message sent to the coroutine runner. It turns 
out that asyncio only uses awaitable objects for its messages, so this 
code can get away with this on asyncio, but if you try using this code 
with curio then I'm pretty sure you're going to end up doing something 
like "await (3,)" and then blowing up.



PEP 492 clearly states the following:

It is aTypeErrorto pass anything other than an/awaitable/object to 
anawaitexpression.


That (3,) is not an awaitable, so the example is invalid. That said, I 
will re-examine this part of the implementation and correct it if 
necessary. So far I just haven't encountered anything that would produce 
an error.


>> > Native async generators don't support "yield from" either, so I 
didn't try

>> > to add that until there's a new PEP that defines its exact semantics.
>>
>> Yeah, one of the motivations for async_generator has been to figure 
out what the semantics for native generators should be :-). The native 
async generators in 3.6 use the same implementation trick as 
async_generator (not entirely sure if Yury got it from me, or if it's 
an independent invention), and the yield from implementation is partly 
motivated as a proof of concept / testing ground for hopefully getting 
it into 3.7. The yield from semantics are pretty straightforward 
though, I think.

>>
>> > As for
>> > aclose/asend/athrow support, I will have to look into that.
>> >
>> > Could we synchronize the implementations so that they're 
compatible with
>> > each other? I would've preferred there to be only a single 
implementation
>> > (asyncio_extras was published first, but admittedly I didn't 
advertise it
>> > that well) but now it'd be best if we work together, wouldn't you 
agree?

>>
>> Makes sense to me :-). I think the obvious approach would be for 
async_extras to just depend on async_generator, since AFAICT 
async_generator is pretty much complete, and like you say, there's not 
much point in carrying around two copies of the same thing. But if you 
have another suggestion I'd be interested to hear it...

>
> I may consider that at some point, but I'm not yet comfortable with 
those extended capabilities. Meanwhile I will add any missing 
functionality (asend, athrow etc.) to mine.


You could just not import yield_from_ and then you don't have any 
extended capabilities... but up to you, obviously.


> One thing I noticed is that there seems to be no way to detect async 
generator functions in your implementation. That is something I would 
want to have before switching.


Good point. That should be pretty trivial to add.

-n



___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-sig
Code of Conduct: https://www.python.org/psf/codeofconduct/