[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: My hypothetical "operator.getfuture()" would be a functional spelling of "what an await expression does to retrieve a future from an awaitable object". Whether that's actually useful is an open question, hence deferring the idea to 3.6 at the earliest :)

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Ben Darnell added the comment: I don't think operator.getfuture() is possible because there are multiple ways of turning an awaitable into a Future. asyncio has one way; tornado has another. -- ___ Python tracker

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > I think we should add "inspect.isawaitable()", and have it pass for > *anything* that can be used in an await expression (whether that's by > implementing the Awaitable protocol, or due to special casing at the > interpreter level). I agree. I've committed

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset cb1aafc9ad7e by Yury Selivanov in branch '3.5': Issue #24400: Resurrect inspect.isawaitable() https://hg.python.org/cpython/rev/cb1aafc9ad7e New changeset a14f6a70d013 by Yury Selivanov in branch 'default': Merge 3.5 (Issue #24400) https://hg.python

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: Sorry, I forgot to state my main conclusion in comparing the consequences of adding "inspect.isawaitable" vs adding an API specifically for checking "isn't Awaitable, but can be used in an await expression": I think we should add "inspect.isawaitable()", and

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: "might be useful in the future" is an API design red flag that suggests to me we may not know what "good" looks like here yet. At the same time, we need something Tornado et al can use to accept the "can be used with await expressions, but doesn't implement the

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > If you want to cover the "iterable coroutine" case, why not add an inspect > helper function for that? That's clearly a concrete type (even more > concrete than a concrete type) that can be inspected. Because I view "iterable coroutines" as a temporary, transi

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: > 1. abc.Coroutine and abc.Awaitable will guarantee that objects that implement > them have '__await__' and it's safe to access it (that means that they will > fail for generator-based coroutines). Absolutely. That was the main theme behind the whole type split

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Guido, Ben, Stefan, Nick, I want to re-open this issue. I'm still not satisfied with the current state of things, mainly with the __instancecheck__ hack for Awaitable & Coroutine ABCs (as Ben initially suggested). I think we should remove the __instancecheck

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Thanks, Martin, it was indeed something that shouldn't been committed. -- ___ Python tracker ___ ___

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset 68996acdec6f by Yury Selivanov in branch '3.5': docs/conf: Undo changes in b2a3baa1c2b0; issue #24400 https://hg.python.org/cpython/rev/68996acdec6f -- ___ Python tracker

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Martin Panter added the comment: The last change to /Doc/conf.py seems to have screwed up my docs build. Was that an accident? $ make -C Doc/ htmlsphinx-build -b html -d build/doctrees -D latex_paper_size= . build/html Running Sphinx v1.2.3 loading pickled environment... done Theme error: n

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset b2a3baa1c2b0 by Yury Selivanov in branch '3.5': Issue #24400: Mention that __instancecheck__ is used in abc.Awaitable and Coroutine https://hg.python.org/cpython/rev/b2a3baa1c2b0 New changeset 4bf1d332fe73 by Yury Selivanov in branch 'default': Mer

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset a9d38701536d by Yury Selivanov in branch '3.5': Issue #24400: Add one more unittest for CoroutineType.__await__ https://hg.python.org/cpython/rev/a9d38701536d -- ___ Python tracker

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Yury Selivanov : -- status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://m

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Martin Panter added the comment: Opened Issue 24541 related to this latest change. The test and documentation are still inconsistent, even if the test passes. -- ___ Python tracker

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > Look like you forgot to adjust test_inspect for the removal. eg: My bad. Thanks, David! -- ___ Python tracker ___ ___

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset 0b7c313851ca by Yury Selivanov in branch '3.5': Issue #24400: Fix failing unittest https://hg.python.org/cpython/rev/0b7c313851ca New changeset 8c85291e86bf by Yury Selivanov in branch 'default': Merge 3.5 (Issue #24400) https://hg.python.org/cpytho

[issue24400] Awaitable ABC incompatible with functools.singledispatch

R. David Murray added the comment: Look like you forgot to adjust test_inspect for the removal. eg: http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Non-Debug%203.x/builds/54 -- nosy: +r.david.murray status: closed -> open ___ Python trac

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset e20c197f19d6 by Yury Selivanov in branch '3.5': Issue #24400: Remove inspect.isawaitable(). https://hg.python.org/cpython/rev/e20c197f19d6 New changeset 800bf6a0e0d5 by Yury Selivanov in branch 'default': Merge 3.5 (Issue #24400) https://hg.python.o

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Ben Darnell added the comment: Yes, I can switch use the ABC instead, and I agree that it doesn't make sense to have the inspect method if it's going to be equivalent to the ABC. I'm happy with the outcome here but AFAIK the original issue still stands: the Awaitable ABC is unusual in that `is

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > > isawaitable(), however, should continue using abc.Awaitable, since it only > > checks for __await__ presence on the type (or should we just drop it?) > I'd really remove it. It's not referring to an actual type, so it doesn't fit > the purpose of the inspe

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: > isawaitable(), however, should continue using abc.Awaitable, since it only > checks for __await__ presence on the type (or should we just drop it?) I'd really remove it. It's not referring to an actual type, so it doesn't fit the purpose of the inspect module

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: > I originally planned to make the next Cython release patch the Generator > and Coroutine ABCs into collections.abc, but I now think it would be worth > uploading an "abc_backports" package to PyPI instead that does that and on > which asyncio, tornado and other

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset 9aee273bf8b7 by Yury Selivanov in branch '3.5': Issue #24400, #24325: More tests for types._GeneratorWrapper https://hg.python.org/cpython/rev/9aee273bf8b7 New changeset fa097a336079 by Yury Selivanov in branch 'default': Merge 3.5 (issue #24325 & #

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset eb6fb8e2f995 by Yury Selivanov in branch '3.5': Issue #24325, #24400: Add more unittests for types.coroutine; tweak wrapper implementation. https://hg.python.org/cpython/rev/eb6fb8e2f995 New changeset 7a2a79362bbe by Yury Selivanov in branch 'defau

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset 8f4e738cb07f by Yury Selivanov in branch '3.5': Issue #24495, #24400: Test asyncio.Task.repr in debug mode https://hg.python.org/cpython/rev/8f4e738cb07f -- ___ Python tracker

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset 3a78be4bcbde by Yury Selivanov in branch '3.4': Issue #24400: Fix CoroWrapper for 'async def' coroutines https://hg.python.org/cpython/rev/3a78be4bcbde New changeset 338597d2e93b by Yury Selivanov in branch '3.5': Issue #24400: Fix CoroWrapper for '

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset f4b702672beb by Yury Selivanov in branch '3.4': asyncio: Merge changes from issue #24400. https://hg.python.org/cpython/rev/f4b702672beb -- ___ Python tracker

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Nick, Martin, Stefan, thanks to all of you for the code review! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker _

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Roundup Robot added the comment: New changeset 7a0a1a4ac639 by Yury Selivanov in branch '3.5': Issue #24400: Introduce a distinct type for 'async def' coroutines. https://hg.python.org/cpython/rev/7a0a1a4ac639 New changeset 44253ce374fc by Yury Selivanov in branch 'default': Issue #24400: Merge

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: An updated patch is attached. I'll commit it tomorrow morning. -- Added file: http://bugs.python.org/file39764/corotype.patch ___ Python tracker ___

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: I added three more comments to the review. The rest looks good to me, too. -- ___ Python tracker ___

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: Latest version looks good to me - I'd suggest we merge this, and file any remaining problems as separate issues. -- ___ Python tracker ___ __

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Nick, Martin, here's an updated patch with all your comments addressed. -- Added file: http://bugs.python.org/file39751/corotype.patch ___ Python tracker __

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: I believe the key issue of concern for Cython is being able to emulate the native CPython types in a way that things like asyncio "just work", rather than specifically needing to have inspect report them as native generators and coroutines. As such, having insp

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: OK, let's go with cr_* prefix for coroutine type's slots. Nick, there is one more thing I'd be glad to receive your input on. Currently, inspect.iscoroutine(o) uses isinstance(o, abc.Coroutine) to do the check. Ben and Stefan reasonably ask to refactor isgen

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: Based on that last comment, I realised there's something else @types.coroutine should now do: delegate both the existing gi_* attributes *and* the new cr_* attributes to the underlying generator. That should allow all of the poking around in code object flags to

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: My view is that the "gi_*" prefixed attributes are the ducktyped representation of whether an object is a generator-iterator or a coroutine. With the change, it means "hasattr(obj, 'cr_running')" is to "hasattr(obj, 'gi_running') as "isinstance(obj, collections.

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Yury Selivanov : Added file: http://bugs.python.org/file39745/corotype.patch ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Actually, I'm not sure that we should use 'cr_*' prefix instead of 'gi_*' for coroutines. Coroutines reusing generators machinery is a two-fold thing: on the one hand it makes the implementation simpler; on the other -- __await__ must return an *iterator*. I

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: New patch is attached. Updates: 1. Coroutine type now has 'cr_*' slots *both* in Python *and* in C. 2. set_coroutine_wrapper now works *only* on coroutines created by 'async def' functions (generators wrapped with types.coroutine won't be intercepted). 3. Ru

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: In my last set of review comments, I suggested changing the Python level attributes for coroutine objects to cr_frame, cr_code, and cr_running. That reminded me that now that coroutines are their own type, we should also give them their own state introspection A

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Yury Selivanov : Added file: http://bugs.python.org/file39730/corotype.patch ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Another iteration of the patch is attached. Nick, I think it's ready for your review. -- stage: needs patch -> patch review type: -> enhancement Added file: http://bugs.python.org/file39729/corotype.patch ___ Pytho

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Yury Selivanov : Added file: http://bugs.python.org/file39723/corotype.patch ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Yury Selivanov : Added file: http://bugs.python.org/file39722/corotype.patch ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Nick, Stefan, Martin, Please find an updated patch attached. All issues that you guys found (thanks a lot for the reviews btw!) should be resolved. -- Added file: http://bugs.python.org/file39720/corotype.patch __

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Martin Panter added the comment: One problem with 2015-06-10’s patch (x86-64 Linux): >>> async def f(): ... pass ... >>> c = f() >>> w = c.__await__() >>> w >>> dir(w) TypeError: object does not provide __dir__ >>> type(w) Segmentation fault (core dumped) [Exit 139] Strangely, if I call w

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Martin Panter : -- nosy: +vadmium ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: Also worth noting: *adding* state to coroutines that isn't present on generators would be fine, they just need to *start* with a layout that exactly matches PyGenObject, since we'll sometimes process them as one. We don't need that now, but separating the types

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: I agree that a typedef is a good idea. It doesn't cost anything but gives us more freedom for future changes. -- ___ Python tracker ___ _

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: Regarding the idea of doing a typedef for the new coro type at the C level: looking further at the way the new type integrates with the eval loop, it's essential that they actually retain the exact same memory layout if we don't want to rewrite a whole lot of co

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: Low level review sent. Regarding the new opcode, it appears the main thing it provides is early detection of yielding from a coroutine in a normal generator, which is never going to work properly (a for loop or other iterative construct can't drive a coroutine

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Guido van Rossum added the comment: FYI I am on vacation and don't have the bandwidth to look into this, so I hope you will all work together to find a solution without my help. -- ___ Python tracker __

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: > I currently do isinstance(x, types.GeneratorType), which will fail if x > is actually a GeneratorWrapper. I can change this to use > collections.abc.Generator when it exists, but then I'd have to have some > conditional logic to switch between collections.abc an

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > With the two changes I described things appear to be working, although I've > only done light testing so far. Glad to hear that! I've attached a new patch fixing types.coroutine per your request. > More generally, the inconsistency between isgenerator() an

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Ben Darnell added the comment: With the two changes I described things appear to be working, although I've only done light testing so far. For inspect.isgenerator(), my use case is here: https://github.com/tornadoweb/tornado/blob/2971e857104f8d02fa9107a0e13f50170eb4f30d/tornado/gen.py#L222 I

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: A couple of high level observations: 1. As Yury notes, more clearly separating coroutines and generators is the intent of the PEP. The computer could clearly cope with them being the same class, but humans tended to get confused. We were still blurring that line t

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > GeneratorWrapper helps, but it fails when applied to non-generator functions > that return a value (while both tornado.gen.coroutine and asyncio.coroutine > take pains to support such usage). The "raise TypeError" should be removed; > just return the result

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Ben Darnell added the comment: GeneratorWrapper helps, but it fails when applied to non-generator functions that return a value (while both tornado.gen.coroutine and asyncio.coroutine take pains to support such usage). The "raise TypeError" should be removed; just return the result without wra

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: On June 9, 2015 at 11:11:11 PM, Ben Darnell (rep...@bugs.python.org) wrote: [..] > The type of the callable matters for the types.coroutine decorator. In > order to get a coroutine object instead of a generator object, I must apply > types.coroutine to the actual

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Ben Darnell added the comment: On Tue, Jun 9, 2015 at 10:12 PM, Yury Selivanov wrote: > > Yury Selivanov added the comment: > > > All this checking for coroutine-ness feels very strange to me. It's > anti-duck-typing: [..] > > Why is it "anti-duck-typing"? Awaitable is an object that implement

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > All this checking for coroutine-ness feels very strange to me. It's > anti-duck-typing: [..] Why is it "anti-duck-typing"? Awaitable is an object that implements __await__. With this patch coroutines are a separate type with __await__ (although, ceval doe

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Ben Darnell added the comment: > 4. While this patch addresses initial request from Ben only partially > (generator-based coroutines still require __instancecheck__), A partial solution doesn't mean much to me: as long as the __instancecheck__ is sometimes necessary, I'll have to use inspect.i

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > (although I think the PEP 492 docs in general are still pending, in which > case, this could just be rolled into that) Actually most of pep 492 docs are merged already (including new opcodes) via issue24180. They can be definitely improved though (I'll try t

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: > Nick Coghlan added the comment: > > A quick scan of which files have been modified suggests the new opcode > still needs docs (although I think the PEP 492 docs in general are still > pending, in which case, this could just be rolled into that) Sure. If the p

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Nick Coghlan added the comment: A quick scan of which files have been modified suggests the new opcode still needs docs (although I think the PEP 492 docs in general are still pending, in which case, this could just be rolled into that) I'll review the full patch later today (too big to review o

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Nick, Guido, I think we should commit this. While working on this I'm getting more and more confident that it's the right thing to do. I like that coroutines implement __await__ in the latest patch -- PEP 492 just now "clicks" with PEP 380. --

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Please find attached a new patch. Stefan, while working on the patch, I (re-)discovered that __await__ for coroutines should return an iterator that also implements '.send', '.throw', and '.close', to comply with PEP 380 yield from implementation: https://www

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: I added some review comments. The main thing is that the coroutine type should be awaitable. For reference, here's my current implementation for Cython. It's a bit more involved as it needs to support Python 2.6+. I may also add some special casing for CPython

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Andrew Svetlov : -- nosy: +asvetlov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyt

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: One more patch fixing minor bug in types.coroutine + a unittest for that. The patch should be ready for reviews. -- Added file: http://bugs.python.org/file39666/corotype.patch ___ Python tracker

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Yury Selivanov : -- hgrepos: +312 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Cleaned up the patch a little bit. -- Added file: http://bugs.python.org/file39665/corotype.patch ___ Python tracker ___ ___

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: Looks good, simplifies the code visibly and makes the implementation quite similar to the one I wrote for Cython. I like the fact that it separates generators from coroutines at an isinstance() level. The fact that "async def" functions currently have the same

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: Please see the attached patch. Couple of notes: 1. It adds a new type: 'coroutine' (PyCoro_Type). The implementation is still heavily based on generators, there are no changes there. 2. Unfortunately, a new opcode needs to be added - GET_YIELD_FROM_ITER. The

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: I'll have a patch soon. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Stefan Behnel added the comment: I agree that the "extra bit" is a quirk. Ideally, Coroutine and Generator would share a common base class that has "send", "throw" and "close". Additionally, a Coroutine would be an Awaitable whereas a Generator is an Iterator. The internal implementation of ge

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Yury Selivanov added the comment: I think that the only proper way to solve this is to make coroutines a separate type. I've actually prototyped that before: https://github.com/1st1/cpython/commit/a3f1059590f496bf77b33edb023c8cdbc1d30798 -- assignee: -> yselivanov components: +Interp

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Changes by Ned Deily : -- priority: normal -> release blocker stage: -> needs patch ___ Python tracker ___ ___ Python-bugs-list maili

[issue24400] Awaitable ABC incompatible with functools.singledispatch

Guido van Rossum added the comment: I think it's actually good feedback and we should fix this during the beta. -- ___ Python tracker ___

[issue24400] Awaitable ABC incompatible with functools.singledispatch

New submission from Ben Darnell: The new collections.abc.Awaitable ABC relies on __instancecheck__, which makes it incompatible with functools.singledispatch (singledispatch works based on args[0].__class__; any instance-level information is discarded). This surprised me because the first thin