[Python-Dev] Enum bug?
Should this be considered a bug in the Enum implementation? >>> class Foo(enum.Enum): ... A = True ... B = 1 ... C = 0 ... D = False ... >>> Foo.A >>> Foo(True) >>> Foo(1) Seems to me like it should store and compare both type and value. Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5IJPHFRLPZE5CGYZH6IXCDH2V4ODXMTB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Dusting off PEP 533?
I've now experienced an issue I believe that PEP 533 was intended to address: When an asynchronous context manager is created within an asynchronous generator, if the generator is not iterated fully, the context manager will not exit until the event loop cancels the task by raising a CancelledError, long after the context manager is assumed to be out of scope. Per PEP 525, I can call aclose coroutine method to cleanup the generator, but it requires the code iterating to be aware that that closing the generator is necessary. Any appetite for putting PEP 533 back on the table to address this kind of issue? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RZXOTQT3QXCVWCPZOWTIJNWTT2TZLWCW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally
My code pretty much does what you suggest at the end of your message: On Mon, 2021-01-11 at 09:22 -0800, Larry Hastings wrote: > Or code can just use inspect.get_type_hints(), which is tied to the > Python version > anyway and should always do the right thing. So far, this has proven mostly[1] sufficient for my needs in a runtime type validation and encoding/decoding library. [1] A pain point for me is the runtime cost of evaluating 3.10 style type hints, as they're (re-)evaluated for every call to get_type_hints. I've worked around this for now with my own function affix_type_hints, which evaluates get_type_hints once and replaces __annotations__ with the evaluated value. It also addresses a scoping problem where a type hint may reference a value that's not globally scoped for the object being annotated; the hint can be evaluated and affixed within that scope. Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/36C5BLHJEBMFUNCC7L5GUHNLHKXH6GGX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
I'm very much in favour of the this concept. A few points come to mind right away: 1. Backwards Compatibility > PEP 563 changed the semantics of annotations. When its semantics are > active, annotations must assume they will be evaluated inmodule-level > scope. They may no longer refer directly to local variables or class > attributes. Given get_type_hints can be provided localns argument, this statement is not exactly true. Using localns is how I currently address scoping issues when affixing type hints. Maybe it could be argued that I'm abusing this feature, but then I would ask what the intent of localns is if not to provide additional (missing) scope during type hint evaluation? Under PEP 649, when __co_annotations__ is called (presumably by calling get_type_hints), would localns effectively be ignored? 2. __co_annotations__ scope? I'm wondering why __co_annotations__ function could not be scoped (within a closure?) such that it can access the values where the function, method, class or module are being declared? I acknowledge that I'm railing against PEP 563 again, trying to reclaim lost ground. On Mon, 2021-01-11 at 10:27 -0800, Larry Hastings wrote: > > On 1/11/21 10:16 AM, Chris Angelico wrote: > > > Number allocation is pretty informal. Go ahead and grab PEP 649; > It's now checked in as PEP 649, with a modern header, modern > copyright, and I went ahead and grabbed the formatting stanza from > the end too. > Welcome to the world, baby 649! > > /arry > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/OKOQEAEPKYDX6AVEFD7DDPBKOHGXB4GB/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5GLDASRAWNEJXYG3Y7NDUBPPNR6TTIGX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
On Mon, 2021-01-11 at 13:16 -0800, Larry Hastings wrote: > > Thanks for your feedback! I'll reply piecemeal. > > On 1/11/21 12:32 PM, Paul Bryan wrote: > > > 1. Backwards Compatibility > > > > > > > PEP 563 changed the semantics of annotations. When its semantics > > > are active, annotations must assume they will be evaluated in > > > module-level scope. They may no longer refer directly to local > > > variables or class attributes. > > > > Given get_type_hints can be provided localns argument, this > > statement is not exactly true. > PEP 563 states: > > > For code that uses type hints, the typing.get_type_hints(obj, > > globalns=None, localns=None) function correctly evaluates > > expressions back from its string form. > So, if you are passing in a localns argument that isn't None, okay, > but you're not using them "correctly" according to the language. > Also, this usage won't be compatible with static type checkers. I acknowledge that this will not fly with static type checkers. I also want to make sure that annotations can continue to serve runtime type validation. PEP 563 does go on to state: > For code which uses annotations for other purposes, a > regulareval(ann, globals, locals) call is enough to resolve the > annotation. And I believe this would no longer be true under PEP 649; further, localns (and maybe globalns) parameters in get_type_hints would become meaningless. This passage in PEP 563 appears not true in Python 3.9 with __future__ annotations, emphasis mine: > The get_type_hints() function automatically resolves the correct > value of globalns for functions and classes. It also automatically > provides the correct localns for classes. If this passage was true, I believe the issue that resulted in my affixing type hints could have been averted. > > Under PEP 649, when __co_annotations__ is called (presumably by > > calling get_type_hints), would localns effectively be ignored? > Yes. You can experiment with this in Python 3.9--just turn off > annotation stringizing. It seems that you can still use strings as > annotations and typing.get_type_hints() will evaluate them--and I > assume it'll use localns at that point, just as it does today. OK, would string representations of type hints continue be supported under PEP 649 if strings are used as annotations? And, would get_type_hints continue evaluate the annotations in that case? > > 2. __co_annotations__ scope? > > > > I'm wondering why __co_annotations__ function could not be scoped > > (within a closure?) such that it can access the values where the > > function, method, class or module are being declared? I acknowledge > > that I'm railing against PEP 563 again, trying to reclaim lost > > ground. > This is addressed in PEP 563, when it rejected the idea of using > "function local state when defining annotations": I wasn't thinking the function local state of that being annotated (agree, this would be prohibitive), but rather the scope in which the annotated function, class, module, etc. are being defined. > > This would be prohibitively expensive for highly annotated code as > > the frames would keep all their objects alive. That includes > > predominantly objects that won't ever be accessed again. > > > > https://www.python.org/dev/peps/pep-0563/#keeping-the-ability-to-use-function-local-state-when-defining-annotations > Doing this automatically would indeed incur a sizeable runtime cost, > for a feature that is already rarely used at runtime. I guess it > would be remotely possible? to add this as an optional feature? But > this gets crazy quickly--what if it's defined inside a function > inside another function inside a class inside another function?--and > the use cases seem few, and TOOWTDI. I think this exactly the case for closures today. > I've never understood how closures work in Python, so I'm not the guy > to ask how possible / hard this would be. Then again, the > implementation of closures is obscure enough that I've never been > able to understand them, so that seems to establish at least a base > level of difficulty. > Anyway, one of the concepts my PEP is built on is that "annotations > are always evaluated at module-level scope". I'd be against changing > that unless it could be achieved without runtime cost--which AFAIK is > impossible. > > Cheers, > > /arry ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZOEQFZ4RPZJW7MWO7US5OH23357QQLNV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
Some more questions... "Binding"," bound" and "unbound" code objects: Is your use of "binding" terminology in the PEP identical to the binding of a function to an object instance as a method during object creation? Function Annotations: > When binding an unbound annotation code object, a function will use > its own __globals__ as the new function's globals. I'm having trouble parsing this. Does this mean the newly bound __co_annotations__ function will inherit __globals__ from the function it's annotating? Exceptions: It's quite possible for a __co_annotation__ function call to raise an exception (e.g. NameError). When accessing __annotations__, if such an exception is raised during the call to __co_annotations__, what is the expected behavior? s/__co_//?: I'm probably naive, but is there a reason that one could not just store a callable in __annotations__, and use the descriptor to resolve it to a dictionary and store it when it is accessed? It would be one less dunder in the Python data model. On Mon, 2021-01-11 at 15:46 -0800, Larry Hastings wrote: > > On 1/11/21 3:02 PM, Paul Bryan wrote: > > > PEP 563 does go on to state: > > > > > For code which uses annotations for other purposes, a regular > > > eval(ann, globals, locals) call is enough to resolve the > > > annotation. > > > > And I believe this would no longer be true under PEP 649; > > further, localns (and maybe globalns) parameters in get_type_hints > > would become meaningless. > > > > [...] > > And, would get_type_hints continue evaluate [string] annotations in > > that case? > I don't work on typing.get_type_hints() so someone else will have to > answer this question. All I can say is, with PEP 649 semantics, if > you set an annotation to a string, you'll get a string back. And in > 3.9 (and my out-of-date 3.10) I observe that typing.get_type_hints() > will eval() string annotations for you, and localns is significant. > > > > This passage in PEP 563 appears not true in Python 3.9 with > > __future__ annotations, emphasis mine: > > > > > The get_type_hints() function automatically resolves the correct > > > value of globalns for functions and classes. It also > > > automatically provides the correct localns for classes. > > > > If this passage was true, I believe the issue that resulted in my > > affixing type hints could have been averted. > As you've discovered, this is one of the places where PEP 563 seems > to be out-of-date with respect to its implementation. I sifted > through the source code to typing.get_type_hints() twice, and near as > I can figure out, localns is literally only ever set to None unless > you override it with the parameter. > > > > OK, would string representations of type hints continue be > > supported under PEP 649 if strings are used as annotations? > PEP 649 is itself totally agnostic as to what value you use as an > annotation. It disallows a couple funky things (yield, walrus > operator), but beyond that it doesn't care. Any Python expression or > value is fine. > > > > > > > > > > > 2. __co_annotations__ scope? > > > > > > > > I'm wondering why __co_annotations__ function could not be > > > > scoped (within a closure?) such that it can access the values > > > > where the function, method, class or module are being declared? > > > > I acknowledge that I'm railing against PEP 563 again, trying to > > > > reclaim lost ground. > > > This is addressed in PEP 563, when it rejected the idea of using > > > "function local state when defining annotations": > > > > I wasn't thinking the function local state of that being annotated > > (agree, this would be prohibitive), but rather the scope in which > > the annotated function, class, module, etc. are being defined. > That's what PEP 563 is referring to. If you read the thread from > November 2017 where the idea was discussed, they were talking about > referring to e.g. "class-level definitions", as in, things defined > inside class scope. Which is prohibitive. > (If I understand you correctly, you thought it was referring to the > scope inside the function when it runs? Because I can't imagine how > that would ever work. What if the function hasn't been called yet? > What if it's been called a thousand times? What if it's running > right now in various stages of completeness in five threads and you > inspect the annotation from a sixth thread?) > > Cheers, > > /arry ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4EB5ZQQ65KLH3LDRLM4N4BQMVWGPKAAW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
On Mon, 2021-01-11 at 17:56 -0800, Larry Hastings wrote: > On 1/11/21 5:02 PM, Paul Bryan wrote: > > > > > Some more questions... > > > > "Binding"," bound" and "unbound" code objects: > > Is your use of "binding" terminology in the PEP identical to the > > binding of a function to an object instance as a method during > > object creation? > I'm not. In PEP 649 I think every reference of "binding" is talking > about binding a code object to a globals dict to produce a function > object. The process of binding a function to an object instance to > make a method is conceptually very similar, but distinct. > (and btw, functions aren't bound to their object to make methods > during object creation, it's done lazily at the time you ask for it-- > that's what the "descriptor protocol" is all about!) I think it'd be worth briefly describing binding, to avoid confusion. I'm now more educated on function/method binding lifecycle though, so bonus! 🙂 > > Function Annotations: > > > > > When binding an unbound annotation code object, a function will > > > use its own __globals__ as the new function's globals. > > I'm having trouble parsing this. Does this mean the newly bound > > __co_annotations__ function will inherit __globals__ from the > > function it's annotating? > Yes. Though I wouldn't use "inherit", I'd just say it "uses" the > __globals__ from the function. Then I think this solves my particular scoping problem with 3.10 string-evaluated annotations! > > Exceptions: > > It's quite possible for a __co_annotation__ function call to raise > > an exception (e.g. NameError). When accessing __annotations__, if > > such an exception is raised during the call to __co_annotations__, > > what is the expected behavior? > If the function fails for any reason--throws an exception, or just > doesn't return an acceptable value--then the getter immediately > exits, and the internal state of the object is unchanged. If you > wanted to, you could catch the exception, fix the error, and get > __annotations__ again, and it'd work. OK. > > s/__co_//?: > > I'm probably naive, but is there a reason that one could not just > > store a callable in __annotations__, and use the descriptor to > > resolve it to a dictionary and store it when it is accessed? It > > would be one less dunder in the Python data model. > That would work, but I think the API is a bit of a code smell. > __annotations__ would no longer be stable: > > a.__annotations__ = o > > assert a.__annotations__ == o > Would that assert fail? It depends on what type(o) is, which is > surprising. Equally surprising?: a.__co_annotations__ = o a.__annotations__ assert a.__co_annotations__ == o Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Z7SFYJZQHUQKZC77L4EMNZIYAQUXK3GJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
Will do. On Mon, 2021-01-11 at 20:55 -0800, Guido van Rossum wrote: > On Mon, Jan 11, 2021 at 3:51 PM Larry Hastings > wrote: > > [...] > > > This passage in PEP 563 appears not true in Python 3.9 with > > > __future__ annotations, emphasis mine: > > > > > > > The get_type_hints() function automatically resolves the > > > > correct value of globalns for functions and classes. It also > > > > automatically provides the correct localns for classes. > > > > > > If this passage was true, I believe the issue that resulted in my > > > affixing type hints could have been averted. > > As you've discovered, this is one of the places where PEP 563 seems > > to be out-of-date with respect to its implementation. I sifted > > through the source code to typing.get_type_hints() twice, and near > > as I can figure out, localns is literally only ever set to None > > unless you override it with the parameter. > > > > > This seems to be a bug in get_type_hints() for which someone should > file a bug on bpo, please! ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LS36RQ2UPLFKIOJAXF7EAGEMKBOCEIKW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
On Tue, 2021-01-12 at 09:54 -0800, Larry Hastings wrote: > Note that this only works in the current version of the PEP / > prototype, where annotations are strictly evaluated in module scope. > If we start supporting closures, those need "cell" objects, which > IIUC can't be marshalled. Since the __co_annotations__ function will get globals from the function it annotates, doesn't it get more than just module scope? Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VWY42HWKA5LQ7BZ7VROCEDWVYHWELSBI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally
On Tue, 2021-01-12 at 20:09 -0800, Guido van Rossum wrote: > On Tue, Jan 12, 2021 at 8:00 PM Brett Cannon > wrote: > > > > > * It turns a None annotation into type(None). Which means now > > > you > > > can't tell the difference between "None" and "type(None)". > > > > > Huh, I wasn't aware of that. > > > > This has tripped up many people. Maybe we should just bite the bullet > and change this? +1, FWIW. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GBM2Z3TN42LR4DKMPMDXSD7WPKWGBFDW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
OK. Makes sense to think of __annotations__ as being the location of the final, stable, "affixed" type hints. I wonder if then the __co_annotations__ call and overwriting of __annotations__ should be explicitly caused by a to get_type_hints instead of (mysteriously) occurring on an attempt to getattr __annotations__. I know this changes the descriptor behavior you documented, but at least it would occur explicitly in a function call and may be easier for developers to reason about? It would also address my other question of trying to access __annotations__, only to be confronted with an exception raised within __co_annotations__. On Fri, 2021-01-15 at 09:47 -0800, Larry Hastings wrote: > > On 1/11/21 6:34 PM, Paul Bryan wrote: > > On Mon, 2021-01-11 at 17:56 -0800, Larry Hastings wrote: > > > > On 1/11/21 5:02 PM, Paul Bryan wrote: > > > > > > > > I'm probably naive, but is there a reason that one could not just > > > store a callable in __annotations__, and use the descriptor to > > > resolve it to a dictionary and store it when it is accessed? It > > > would be one less dunder in the Python data model. > > That would work, but I think the API is a bit of a code smell. > > __annotations__ would no longer be stable: > > > > > a.__annotations__ = o > > > assert a.__annotations__ == o > > Would that assert fail? It depends on what type(o) is, which is > > surprising. > > Equally surprising?: > > a.__co_annotations__ = o > a.__annotations__ > assert a.__co_annotations__ == o > > I've ruminated about this a bit over the past few days, and I finally > realized exactly why, yes, I think behavior is more surprising. It's > because __annotations__ is now 12 years old (!), and never in that > entire time has it silently changed its value. It's always been > completely stable, and we have twelve years' worth of installed base > that may rely on that assumption. In comparison, __co_annotations__ > is a new attribute. While it's also surprising that > __co_annotations__ can be automatically unset, at least this would be > a documented part of its behavior from day 1. > Relatedly, __co_annotations__ is behaving somewhat like a cached > value, in that cached values get deleted when they're out-of-date. > (An observation that may provide some guidance if we decide to rename > __co_annotations__.) This idiom may be familiar to the user--unlike > your proposed semantics, which I don't recall ever seeing used in an > API. > I admit it's only a small difference between what you proposed and > what I propose, but in the end I definitely prefer my approach. > Cheers, > > /arry ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AV6JRZZDDN4GAPE5MHRFEBFUY3TEO7VZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
I knew I was missing something. Agree, annotations are not necessarily type hints. On Fri, 2021-01-15 at 10:56 -0800, Larry Hastings wrote: > > On 1/15/21 10:12 AM, Paul Bryan wrote: > > > I wonder if then the __co_annotations__ call and overwriting of > > __annotations__ should be explicitly caused by a to get_type_hints > > instead of (mysteriously) occurring on an attempt to getattr > > __annotations__. > > I would say: absolutely not. While all "type hints" are annotations, > not all annotations are "type hints". As mentioned previously in > this thread, typing.get_type_hints() is opinionated in ways that > users of annotations may not want. And personally I bristle at the > idea of gating a language feature behind a library function. > Besides, most users will never know or care about > __co_annotations__. If you're not even aware that it exists, it's > not mysterious ;-) > > Cheers, > > /arry ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C3HINI65UIFCA4EON55WMR6YR6THXQCQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors
Would annotations() just access the dunder, like other builtins (and then result in the descriptor resolving __co_annotations__ as proposed), or would calling it be required to actually resolve __co_annotations__? I think it should probably be the former. On Sat, 2021-01-16 at 12:29 +1300, Greg Ewing wrote: > On 16/01/21 7:56 am, Larry Hastings wrote: > > > > As mentioned previously in this > > thread, typing.get_type_hints() is opinionated in ways that users > > of > > annotations may not want. > > This brings us back to my idea of introducing a new > annotations() function to hide the details. It wouldn't > be the same as get_type_hints(), since it wouldn't make > any assumptions about what the annotations mean. > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/457EYGAJAE4BQHUWH32MRZNSLFGNR6MM/ Code of Conduct: http://python.org/psf/codeofconduct/
