[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Another thought about this PEP (hopefully my last one tonight). The section on backwards compatibility doesn't mention what should happen with annotations that are stringified by the user (as is needed for forward references in code that hasn't been PEP-563-ified yet). That's a PEP 484 feature.

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Will do. On Mon, 2021-01-11 at 20:55 -0800, Guido van Rossum wrote: > On Mon, Jan 11, 2021 at 3:51 PM Larry Hastings > wrote: > > [...] > > > This passage in PEP 563 appears not true in Python 3.9 with > > > __future__ annotations, emphasis mine: > > > > > > > The get_type_hints() function

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Mon, Jan 11, 2021 at 3:51 PM Larry Hastings wrote: > [...] > > This passage in PEP 563 appears not true in Python 3.9 with __future__ > annotations, emphasis mine: > > The get_type_hints() function automatically resolves the correct value of > globalns for functions and classes. *It also

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Mon, Jan 11, 2021 at 1:20 PM Larry Hastings wrote: > PEP 563 states: > > For code that uses type hints, the typing.get_type_hints(obj, > globalns=None, localns=None) function correctly evaluates expressions back > from its string form. > > So, if you are passing in a localns argument that

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 6:34 PM, Paul Bryan wrote: a.__annotations__ = o assert a.__annotations__ == o Would that assert fail?  It depends on what type(o) is, which is surprising. Equally surprising?: a.__co_annotations__ = o a.__annotations__ assert a.__co_annotations__ == o Well, since you

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

Oh, but the behavior of annotations in e.g. mypy is the same. They are cumulative. On Mon, Jan 11, 2021 at 17:42 Larry Hastings wrote: > > On 1/11/21 5:28 PM, Guido van Rossum wrote: > > On Mon, Jan 11, 2021 at 5:21 PM Larry Hastings wrote: > >> Slots intelligently support inheritance, too. I

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 6:29 PM, Greg Ewing wrote: On 12/01/21 10:41 am, Larry Hastings wrote: So: if you're using annotations for something besides "type hints", Didn't Guido decree that using annotations for anything other than type hints is no longer supported? It was never an official decree. 

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 1/11/21 6:31 PM, Greg Ewing wrote: On 12/01/21 2:21 pm, Larry Hastings wrote: Slots intelligently support inheritance, too. Are you sure about that? My experiments suggest that it has the same problem as __annotations__: Python 3.8.2 (default, Mar 23 2020, 11:36:18) [Clang 8.1.0

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 12/01/21 2:56 pm, Larry Hastings wrote: In PEP 649 I think every reference of "binding" is talking about binding a code object to a globals dict to produce a function object.   The process of binding a function to an object instance to make a method is conceptually very similar, but

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 12/01/21 2:21 pm, Larry Hastings wrote: Slots intelligently support inheritance, too. Are you sure about that? My experiments suggest that it has the same problem as __annotations__: Python 3.8.2 (default, Mar 23 2020, 11:36:18) [Clang 8.1.0 (clang-802.0.42)] on darwin Type "help",

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 12/01/21 2:46 pm, Larry Hastings wrote: Using an 64-byte empty dict per object with no defined annotations seems so wasteful.  And anything short of an empty dict, you'd have to guard against. If __annotations__ were to return a read-only mapping object instead of a dict, the same empty

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Mon, 2021-01-11 at 17:56 -0800, Larry Hastings wrote: > On 1/11/21 5:02 PM, Paul Bryan wrote: > > > > > Some more questions... > > > > "Binding"," bound" and "unbound" code objects: > > Is your use of "binding" terminology in the PEP identical to the > > binding of a function to an object

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 12/01/21 11:32 am, Łukasz Langa wrote: EdgeDB uses stringified annotations exclusively which minimizes runtime memory usage of annotations because those strings are pretty much all ASCII and many can be interned. 946 -> s_schema.Schema 362 -> str 298 -> sd.CommandContext Seems to me

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 12/01/21 10:41 am, Larry Hastings wrote: So: if you're using annotations for something besides "type hints", Didn't Guido decree that using annotations for anything other than type hints is no longer supported? -- Greg ___ Python-Dev mailing

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 1/11/21 6:09 PM, Chris Angelico wrote: On Tue, Jan 12, 2021 at 12:56 PM Larry Hastings wrote: It was a balancing act. Using an 64-byte empty dict per object with no defined annotations seems so wasteful. And anything short of an empty dict, you'd have to guard against. Current code

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On Tue, Jan 12, 2021 at 12:56 PM Larry Hastings wrote: > > It was a balancing act. Using an 64-byte empty dict per object with no > defined annotations seems so wasteful. And anything short of an empty dict, > you'd have to guard against. Current code already has to guard against >

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 5:02 PM, Paul Bryan wrote: Some more questions... "Binding"," bound" and "unbound" code objects: Is your use of "binding" terminology in the PEP identical to the binding of a function to an object instance as a method during object creation? I'm not.  In PEP 649 I think every

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 1/11/21 5:05 PM, Greg Ewing wrote: On 12/01/21 6:22 am, Larry Hastings wrote:  * The language will set __annotations__ to a dict if the object has    annotations, or None if it has no annotations. That sounds inconvenient -- it means that any code referencing __annotations__ has to guard

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 1/11/21 5:28 PM, Guido van Rossum wrote: On Mon, Jan 11, 2021 at 5:21 PM Larry Hastings > wrote: Slots intelligently support inheritance, too.  I always kind of wondered why annotations didn't support inheritance--if D is a subclass of C, why doesn't

[Python-Dev] Re: __init_subclass__ and metaclasses

On 1/7/21 4:56 AM, Nick Coghlan wrote: > Both EnumMeta and ABCMeta should probably be relying on `__set_name__` > for their per-member set up work these days, rather than deferring > that work until after __new__ returns. And here I was thinking that `__set_name__` was for, well, setting the

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On Mon, Jan 11, 2021 at 5:21 PM Larry Hastings wrote: > > On 1/11/21 4:39 PM, Guido van Rossum wrote: > > The easiest thing would be just to create an empty `__annotations__` for > classes that have no annotated variables, and to hell with the cost. > > I assume you'd keep the existing behavior

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 4:55 PM, Greg Ewing wrote: On 12/01/21 6:21 am, Larry Hastings wrote: Unbound code objects ...The "annotation code object" is then stored *unbound* as the internal value of ``__co_annotations__``; it is then bound on demand when the user asks for

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

> Performance > --- > > Performance with this PEP should be favorable. In general, > resources are only consumed on demand—"you only pay for what you use". > Nice! > There are three scenarios to consider: > > * the runtime cost when annotations aren't defined, > * the runtime cost when

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 12/01/21 10:16 am, Larry Hastings wrote: This is addressed in PEP 563, when it rejected the idea of using "function local state when defining annotations": This would be prohibitively expensive for highly annotated code as the frames would keep all their objects alive. That includes

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 1/11/21 4:39 PM, Guido van Rossum wrote: The easiest thing would be just to create an empty `__annotations__` for classes that have no annotated variables, and to hell with the cost. I assume you'd keep the existing behavior where functions lazy-create an empty dict if they have no

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

Isn't that just typing.get_type_hints()? On Mon, Jan 11, 2021 at 5:11 PM Greg Ewing wrote: > On 12/01/21 6:22 am, Larry Hastings wrote: > > * The language will set __annotations__ to a dict if the object has > >annotations, or None if it has no annotations. > > That sounds inconvenient --

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 2:32 PM, Łukasz Langa wrote: 1. What do you anticipate the memory usage will look like for your solution compared to PEP 563? It depends on the scenario.  I talk about three runtime scenarios in PEP 649.  But the most relevant scenario is "annotations are defined but never

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On 12/01/21 6:22 am, Larry Hastings wrote: * The language will set __annotations__ to a dict if the object has   annotations, or None if it has no annotations. That sounds inconvenient -- it means that any code referencing __annotations__ has to guard against the possibility of it being

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Some more questions... "Binding"," bound" and "unbound" code objects: Is your use of "binding" terminology in the PEP identical to the binding of a function to an object instance as a method during object creation? Function Annotations: > When binding an unbound annotation code object, a

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 12/01/21 6:21 am, Larry Hastings wrote: Unbound code objects ...The "annotation code object" is then stored *unbound* as the internal value of ``__co_annotations__``; it is then bound on demand when the user asks for ``__annotations__``. This seems like premature

[Python-Dev] Re: 3.10 change (?) for __bool__

Ah never mind. Seems to be a real bug -- thanks for reporting! On Mon, Jan 11, 2021 at 2:57 PM Mats Wichmann wrote: > On 1/11/21 1:00 PM, Guido van Rossum wrote: > > All that said (I agree it's surprising that 3.10 seems backwards > > incompatible here) I would personally not raise

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

I think your analysis of the problems is great. I don't worry about people deleting `__attributes__` (and those that do can switch to calling its .clear() method instead) but I worry about people not expecting to get None. So if you can avoid that in your solution that would be great. The easiest

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

At last, a nibble on the other fishing line! ;-) On 1/11/21 1:47 PM, Brett Cannon wrote: So the biggest potential breakages are code that: 1. Directly get the attribute from __dict__ 2. The fact that it would no longer be inherited Am I missing anything else? Those are the big ones, the

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 3:02 PM, Paul Bryan wrote: PEP 563 does go on to state: For code which uses annotations for other purposes, a regular eval(ann, globals, locals) call is enough to resolve the annotation. And I believe this would no longer be true under PEP 649; further, localns (and maybe

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 2021-01-11, Łukasz Langa wrote: > The stringification process which your PEP describes as costly > only happens during compilation of a .py file to .pyc. Since > pip-installing pre-compiles modules for the user at installation > time, there is very little runtime penalty for a fully annotated >

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Mon, 2021-01-11 at 13:16 -0800, Larry Hastings wrote: > > Thanks for your feedback!  I'll reply piecemeal. > > On 1/11/21 12:32 PM, Paul Bryan wrote: > > > 1. Backwards Compatibility > > > > > > > PEP 563 changed the semantics of annotations. When its semantics > > > are active,

[Python-Dev] Re: 3.10 change (?) for __bool__

On 1/11/21 1:00 PM, Guido van Rossum wrote: All that said (I agree it's surprising that 3.10 seems backwards incompatible here) I would personally not raise AttributeError but TypeError in the `__bool__()` method. eh, that was just me picking a cheap something to demo it. the program raises

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

> On 11 Jan 2021, at 18:21, Larry Hastings wrote: > > I've written a new PEP. Please find it below. Happy reading! > Interesting! I like the clever lazy-evaluation of the __annotations__ using a pre-set code object. My only real reservation is that the transition process will be weird

[Python-Dev] Re: 3.10 change (?) for __bool__

I've raised: https://bugs.python.org/issue42899. The triggering commit was: c71581c7a4192e6ba9a79eccc583aaadab300efa bpo-42615: Delete redundant jump instructions that only bypass empty blocks (GH-23733) On Mon, Jan 11, 2021 at 8:04 PM Guido van Rossum wrote: > All that said (I agree it's

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

On Mon, Jan 11, 2021 at 9:23 AM Larry Hastings wrote: > [SNIP - background info] > > > If I could wave my magic wand and do whatever I wanted, I'd change the > semantics for __annotations__ to the following: > > * Functions, classes, and modules always have an __annotations__ member > set. > *

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 1:16 PM, Larry Hastings wrote: On 1/11/21 12:32 PM, Paul Bryan wrote: *1. Backwards Compatibility* PEP 563 changed the semantics of annotations. When its semantics are active, annotations must assume they will be evaluated in

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

The control-flow exclusion is for /module//attribute/ or /class attribute/ annotations: class C:   if random.random() > 0.5:     my_attr:int=3   else:     my_attr2:float=3.5 Your example doesn't define any module attributes or class attributes inside flow control

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Could you be more explicit about what is banned by the control-flow exclusion? I'm assuming that: class A: bar=float if FOO: bar=int def a(x:int, y:int)->int # function defined with annotations inside control flow return x+y

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Thanks for your feedback!  I'll reply piecemeal. On 1/11/21 12:32 PM, Paul Bryan wrote: *1. Backwards Compatibility* PEP 563 changed the semantics of annotations. When its semantics are active, annotations must assume they will be evaluated in

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

I'm very much in favour of the this concept. A few points come to mind right away: 1. Backwards Compatibility > PEP 563 changed the semantics of annotations. When its semantics are > active, annotations must assume they will be evaluated inmodule-level > scope. They may no longer refer directly

[Python-Dev] Re: 3.10 change (?) for __bool__

All that said (I agree it's surprising that 3.10 seems backwards incompatible here) I would personally not raise AttributeError but TypeError in the `__bool__()` method. On Mon, Jan 11, 2021 at 11:51 AM Pablo Galindo Salgado wrote: > This may be related to the changes in

[Python-Dev] Re: 3.10 change (?) for __bool__

This may be related to the changes in https://bugs.python.org/issue42246. Could you open a new issue and add Mark Shannon to it if that turns to be the case? Pablo On Mon, 11 Jan 2021 at 19:36, Chris Angelico wrote: > On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann wrote: > > > > > > On 1/8/21

[Python-Dev] Re: 3.10 change (?) for __bool__

On Tue, Jan 12, 2021 at 6:00 AM Mats Wichmann wrote: > > > On 1/8/21 4:31 PM, Mats Wichmann wrote: > > > > > > Someone reported a testsuite break on stuff I work on (scons) with > > 3.10a4, and it looks similar to this which appears in the changelog at > > > >

[Python-Dev] Re: 3.10 change (?) for __bool__

On 1/8/21 4:31 PM, Mats Wichmann wrote: Someone reported a testsuite break on stuff I work on (scons) with 3.10a4, and it looks similar to this which appears in the changelog at https://docs.python.org/3.10/whatsnew/changelog.html#changelog bpo-23898: Fix inspect.classify_class_attrs() to

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/2021 1:10 PM, Larry Hastings wrote: Certainly.  I'm just another victim in the copy-and-paste wars. I actually have write access to the PEPs repo (I'm a former release manager) so I'd be happy to check it in myself once it gets a number, however that happens.  Before I do so I'll

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 10:29 AM, Barry Warsaw wrote: Given that PEP 563 is now the default in unreleased Python 3.10, does it make sense to introduce yet another __future__ import? What would happen if you just piggybacked your idea onto that change? Part of my proposal is to deprecate PEP 563's

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Thanks for this detailed PEP and analysis, and for the interesting discussion in your separate thread. I’m glad to see this work that we chatted about all that time before has coalesced into a PEP. FYI: For those with write access to the PEPs repo, PEP number assignments are self-serve. Just

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 10:16 AM, Chris Angelico wrote: Number allocation is pretty informal. Go ahead and grab PEP 649; It's now checked in as PEP 649, with a modern header, modern copyright, and I went ahead and grabbed the formatting stanza from the end too. Welcome to the world, baby 649! //arry/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Tue, Jan 12, 2021 at 5:10 AM Larry Hastings wrote: > > > Certainly. I'm just another victim in the copy-and-paste wars. > Ah yes, the Battle of the Clipboard. Iconic, epic, such a glorious engagement! But the casualties were steep. Fortunately we can rebuild. > I actually have write access

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Certainly.  I'm just another victim in the copy-and-paste wars. I actually have write access to the PEPs repo (I'm a former release manager) so I'd be happy to check it in myself once it gets a number, however that happens.  Before I do so I'll study PEP 12 as if it was gonna be on

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Tue, Jan 12, 2021 at 4:22 AM Larry Hastings wrote: > > > I've written a new PEP. Please find it below. Happy reading! > Can this get added to the PEPs repo and assigned a number and such? BTW, the currently preferred wording for the copyright blurb is slightly different. If you're the sole

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

My code pretty much does what you suggest at the end of your message: On Mon, 2021-01-11 at 09:22 -0800, Larry Hastings wrote: > Or code can just use inspect.get_type_hints(), which is tied to the > Python version > anyway and should always do the right thing. So far, this has proven mostly[1]

[Python-Dev] PEP: Deferred Evaluation Of Annotations Using Descriptors

I've written a new PEP.  Please find it below.  Happy reading! //arry/ -- PEP: Title: Deferred Evaluation Of Annotations Using Descriptors Version: $Revision$ Last-Modified: $Date$ Author: Larry Hastings Discussions-To: Python-Dev Status: Draft Type: Standards Track

[Python-Dev] Let's Fix Class Annotations -- And Maybe Annotations Generally

Howdy howdy.  While working on my PEP I stumbled over a lot of behavior by annotations that I found inconsistent and inconvenient.  I think there are several problems here that needs fixing.  This discussion will probably evolve into a PEP, and I'll be happy to steer that process.  But I'm less

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

Hi, There are multiple PEPs covering heap types. The latest one refers to other PEPs: PEP 630 "Isolating Extension Modules" by Petr Viktorin. https://www.python.org/dev/peps/pep-0630/#motivation The use case is to embed multiple Python instances (interpreters) in the same application process, or

[Python-Dev] Re: why include cpython/*.h, Need this macro ? # error "this header file must not be included directly"

On Sun, Jan 10, 2021 at 9:27 PM Serhiy Storchaka wrote: > Because it will not produce compile-time error when you include that > header files directly. > > The division of these declarations on few files is a deep implementation > detail. Header file "cpython/fileobject.h" is not a part of API

[Python-Dev] Re: PEP 642 v3: Explicit patterns for structured pattern matching

Hello, On Tue, 12 Jan 2021 00:11:33 +1000 Nick Coghlan wrote: > On Sun, 10 Jan 2021, 7:37 pm Paul Sokolovsky, > wrote: > > > And I patiently continue this thread, hoping that people whose > > argument would be along the lines of "I teach Python, and I don't > > want to teach my students 2

[Python-Dev] Re: PEP 642 v3: Explicit patterns for structured pattern matching

On Sun, 10 Jan 2021, 7:37 pm Paul Sokolovsky, wrote: > And I patiently continue this thread, hoping that people whose argument > would be along the lines of "I teach Python, and I don't want to teach > my students 2 ways of doing the same thing, and which way use when. Why, > if PEP634 offers

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

Hi Victor, Thank you for looking into these issues. They are very important to HPy too! HPy currently only supports head types for similar reasons to why they are important to sub-interpreters -- their lifecycle can be managed by the Python interpreter and they are not tied to the memory and