[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Sun, 18 Apr 2021, 1:59 am Jelle Zijlstra, wrote: > El sáb, 17 abr 2021 a las 8:30, Nick Coghlan () > escribió:. > >> >> Metaclass __prepare__ methods can inject names into the class namespace >> that the compiler doesn't know about, so yeah, it unfortunately has to be >> conservative and use

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

El sáb, 17 abr 2021 a las 8:30, Nick Coghlan () escribió: > > > On Fri, 16 Apr 2021, 3:14 pm Larry Hastings, wrote: > >> >> Anyway I assume it wasn't "fixable". The compiler would presumably >> already prefer to generate LOAD_GLOBAL vs LOAD_NAME, because LOAD_GLOBAL >> would be cheaper every

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Fri, 16 Apr 2021, 3:14 pm Larry Hastings, wrote: > > Anyway I assume it wasn't "fixable". The compiler would presumably > already prefer to generate LOAD_GLOBAL vs LOAD_NAME, because LOAD_GLOBAL > would be cheaper every time for a global or builtin. The fact that it > already doesn't do so

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/15/21 9:24 PM, Inada Naoki wrote: Unlike simple function case, PEP 649 creates function object instead of code object for __co_annotation__ of methods. It cause this overhead. Can we avoid creating functions for each annotation? As the implementation of PEP 649 currently stands, there

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

> > I will read PEP 649 implementation to find missing optimizations other > than GH-25419 and GH-23056. > I found each "__co_annotation__" has own name like "func0.__co_annotation__". It increased pyc size a little. I created a draft pull request for cherry-picking GH-25419 and GH-23056 and

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/15/21 2:02 PM, Sebastián Ramírez wrote: ## Questions I'm not very familiar with the internals of Python, and I'm not sure how the new syntax for `Union`s using the vertical bar character ("pipe", "|") work. But would PEP 649 still support things like this?: def run(arg: int | str = 0):

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Thanks Brett Cannon for suggesting to get Samuel Colvin (Pydantic) and me, Sebastián Ramírez (FastAPI and Typer) involved in this. TL;DR: it seems to me PEP 649 would be incredibly important/useful for Pydantic, FastAPI, Typer, and similar tools, and their communities. ## About FastAPI,

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I updated the benchmark little: * Added no annotation mode for baseline performance. * Better stats output. https://gist.github.com/methane/abb509e5f781cc4a103cc450e1e7925d ``` # No annotation (master + GH-25419) $ ./python ~/ann_test.py 0 code size: 102967 bytes memory: 181288 bytes unmarshal:

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Thu, Apr 15, 2021 at 11:09 AM Larry Hastings wrote: > > Thanks for doing this! I don't think PEP 649 is going to be accepted or > rejected based on either performance or memory usage, but it's nice to see > you confirmed that its performance and memory impact is acceptable. > > > If I run

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Thanks for doing this!  I don't think PEP 649 is going to be accepted or rejected based on either performance or memory usage, but it's nice to see you confirmed that its performance and memory impact is acceptable. If I run "ann_test.py 1", the annotations are already turned into

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I added memory usage data by tracemalloc. ``` # Python 3.9 w/ old semantics $ python3 ann_test.py 1 code size: 121011 memory: (385200, 385200) unmarshal: avg: 0.3341682574478909 +/- 3.700437551781949e-05 exec: avg: 0.4067857594229281 +/- 0.0006858555167675445 # Python 3.9 w/ PEP 563 semantics $

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I created simple benchmark: https://gist.github.com/methane/abb509e5f781cc4a103cc450e1e7925d This benchmark creates 1000 annotated functions and measure time to load and exec. And here is the result. All interpreters are built without --pydebug, --enable-optimization, and --with-lto. ``` #

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Larry Hastings wrote: > On 4/14/21 1:42 PM, Baptiste Carvello wrote: > > Are there specific annoyances associated with quoting always, apart > > from the 2 more characters? > Yes.  Since the quoted strings aren't parsed by Python, syntax errors in > these strings go undetected until somebody

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Baptiste Carvello wrote: > Le 14/04/2021 à 19:44, Guido van Rossum a écrit : > > No, what I heard is that, since in *most* cases the string quotes are > > not needed, people are surprised and annoyed when they encounter cases > > where they are needed. > Well, I had assumed quotes would be used

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/14/21 1:42 PM, Baptiste Carvello wrote: Are there specific annoyances associated with quoting always, apart from the 2 more characters? Yes.  Since the quoted strings aren't parsed by Python, syntax errors in these strings go undetected until somebody does parse them (e.g. your static

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Wed, 2021-04-14 at 22:42 +0200, Baptiste Carvello wrote: > That's assuming the syntax in the annotations doesn't diverge too > much > from the Python syntax as far as brackets etc are concerned. I must > say > I'm not too worried about typing. But the hypothetic "def foo(prec: > --precision

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Hi, Le 14/04/2021 à 19:44, Guido van Rossum a écrit : > > No, what I heard is that, since in *most* cases the string quotes are > not needed, people are surprised and annoyed when they encounter cases > where they are needed. And if you have a large code base it takes an > expensive run of the

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Hi Larry, On 4/14/21, 1:56 PM, "Larry Hastings" wrote: >My plan was to post it here and see what the response was first. Back in > January, when I posted the first draft, I got some very useful feedback that > resulted in some dramatic changes. This time around, so far, nobody has >

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

My plan was to post it here and see what the response was first. Back in January, when I posted the first draft, I got some very useful feedback that resulted in some dramatic changes.  This time around, so far, nobody has suggested even minor changes.  Folks have just expressed their

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Wed, Apr 14, 2021 at 12:08 PM Guido van Rossum wrote: > Let's just wait for the SC to join the discussion. I'm sure they will, > eventually. > FYI the PEP has not been sent to us via https://github.com/python/steering-council/issues as ready for pronouncement, so we have not started

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Let's just wait for the SC to join the discussion. I'm sure they will, eventually. On Wed, Apr 14, 2021 at 11:12 AM Larry Hastings wrote: > On 4/14/21 10:44 AM, Guido van Rossum wrote: > > besides the cost of closing the door to relaxed annotation syntax, there's > the engineering work of

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Wed, Apr 14, 2021 at 11:03 AM Paul Bryan wrote: > What would you expect get_type_hints(...) to return with relaxed syntax? > Today, for type hint annotations, it returns a type, which I'd argue is an > important feature to preserve (in it or some successor). > It would have to return some

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/14/21 10:44 AM, Guido van Rossum wrote: besides the cost of closing the door to relaxed annotation syntax, there's the engineering work of undoing the work that was done to make `from __future__ import annotations` the default (doing this was a significant effort spread over many commits,

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

What would you expect get_type_hints(...) to return with relaxed syntax? Today, for type hint annotations, it returns a type, which I'd argue is an important feature to preserve (in it or some successor). On Wed, 2021-04-14 at 10:54 -0700, Guido van Rossum wrote: > On Wed, Apr 14, 2021 at 10:47

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Wed, Apr 14, 2021 at 10:47 AM Paul Bryan wrote: > I favour annotations for type hints; the writing's been on the wall for > some time. I think the necessary escape hatch for those using it for other > purposes should be Annotated[Any, ...] (or a similar, nicer alternative). > > Guido, one of

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I favour annotations for type hints; the writing's been on the wall for some time. I think the necessary escape hatch for those using it for other purposes should be Annotated[Any, ...] (or a similar, nicer alternative). Guido, one of the difficulties I'm having is understanding the direction

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Wed, Apr 14, 2021 at 9:42 AM Baptiste Carvello < devel2...@baptiste-carvello.net> wrote: > Hi, > > tl;dr: imho the like or dislike of PEP 563 is related to whether people > intend to learn a second syntax for typing, or would rather ignore it; > both groups should be taken into account. > > Le

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, Apr 13, 2021 at 6:58 PM Inada Naoki wrote: > On Wed, Apr 14, 2021 at 10:44 AM Larry Hastings > wrote: > > > > > > On 4/13/21 1:52 PM, Guido van Rossum wrote: > > > > > > Because typing is, to many folks, a Really Important Concept, and it's > confusing to use the same syntax ("x: blah

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, Apr 13, 2021 at 6:48 PM Larry Hastings wrote: > > On 4/13/21 1:52 PM, Guido van Rossum wrote: > > On Tue, Apr 13, 2021 at 12:32 PM Larry Hastings > wrote: > >> >> On 4/12/21 7:24 PM, Guido van Rossum wrote: >> >> I've been thinking about this a bit, and I think that the way forward is

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Hi, tl;dr: imho the like or dislike of PEP 563 is related to whether people intend to learn a second syntax for typing, or would rather ignore it; both groups should be taken into account. Le 13/04/2021 à 19:30, Guido van Rossum a écrit : > On Tue, Apr 13, 2021 at 9:39 AM Baptiste Carvello >

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/12/21 7:24 PM, Guido van Rossum wrote: To be honest, the most pressing issue with annotations is the clumsy way that type variables have to be introduced. The current convention, `T = TypeVar('T')`, is both verbose (why do I have to repeat the name?) and widely misunderstood (many help

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

It looks like a small subset of PEP 484, syntactically. So it should be fine. Possibly cython might be interested in using a relaxed notation if it is ever introduced, e.g. ‘long long’ or ‘static int’ (for a return type)? On Wed, Apr 14, 2021 at 02:27 Antoine Pitrou wrote: > > For the record,

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

For the record, Cython allows using annotations for typing: https://cython.readthedocs.io/en/latest/src/tutorial/pure.html#pep-484-type-annotations I don't know if they are fully compatible with the type hints we're talking about here. Regards Antoine. On Wed, 14 Apr 2021 10:58:07 +0900

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Wed, Apr 14, 2021 at 10:44 AM Larry Hastings wrote: > > > On 4/13/21 1:52 PM, Guido van Rossum wrote: > > > Because typing is, to many folks, a Really Important Concept, and it's > confusing to use the same syntax ("x: blah blah") for different purposes, in > a way that makes it hard to tell

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/13/21 1:52 PM, Guido van Rossum wrote: On Tue, Apr 13, 2021 at 12:32 PM Larry Hastings > wrote: On 4/12/21 7:24 PM, Guido van Rossum wrote: I've been thinking about this a bit, and I think that the way forward is for Python to ignore the text of

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/13/21 3:28 PM, Terry Reedy wrote: On 4/13/2021 4:21 AM, Baptiste Carvello wrote: Le 12/04/2021 à 03:55, Larry Hastings a écrit : * in section "Interactive REPL Shell": For the sake of simplicity, in this case we forego delayed evaluation. The intention of the code + codeop modules

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/13/2021 4:21 AM, Baptiste Carvello wrote: Le 12/04/2021 à 03:55, Larry Hastings a écrit : * in section "Interactive REPL Shell": For the sake of simplicity, in this case we forego delayed evaluation. The intention of the code + codeop modules is that people should be able to write

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, Apr 13, 2021 at 12:32 PM Larry Hastings wrote: > > On 4/12/21 7:24 PM, Guido van Rossum wrote: > > I've been thinking about this a bit, and I think that the way forward is > for Python to ignore the text of annotations ("relaxed annotation syntax"), > not to try and make it available as

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/12/21 7:24 PM, Guido van Rossum wrote: I've been thinking about this a bit, and I think that the way forward is for Python to ignore the text of annotations ("relaxed annotation syntax"), not to try and make it available as an expression. To be honest, the most pressing issue with

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, Apr 13, 2021 at 9:39 AM Baptiste Carvello < devel2...@baptiste-carvello.net> wrote: > Le 13/04/2021 à 04:24, Guido van Rossum a écrit : > > I've been thinking about this a bit, and I think that the way forward is > > for Python to ignore the text of annotations ("relaxed annotation > >

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Hi Larry, On 4/12/21, 6:57 PM, "Larry Hastings" wrote: Again, by "works on PEP 563 semantics", you mean "doesn't raise an error". But the code has an error. It's just that it has been hidden by PEP 563 semantics. I don't agree that changing Python to automatically hide errors is an

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Hi, Le 12/04/2021 à 03:55, Larry Hastings a écrit : > > I look forward to your comments, 2 reading notes: * in section "Annotations That Refer To Class Variables": > If it's possible that an annotation function refers > to class variables--if all these conditions are true: > > * The

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

Hi, Le 13/04/2021 à 04:24, Guido van Rossum a écrit : > I've been thinking about this a bit, and I think that the way forward is > for Python to ignore the text of annotations ("relaxed annotation > syntax"), not to try and make it available as an expression. Then, what's wrong with quoting?

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Mon, 2021-04-12 at 19:52 -0700, Guido van Rossum wrote: > Why not submit a PR that adds caching to get_type_hints(), rather > than promote a paradigm shift? A couple of reasons: 1. In reviewing the code, I didn't find an obvious way to store cached values. Anything but a non-trivial change

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Mon, Apr 12, 2021 at 7:47 PM Paul Bryan wrote: > In 3.9 this cost is paid once when a type is defined. However, in 3.10, it > gets expensive, because when the string is evaluated by get_type_hints, its > result is not stored/cached anywhere (repeated calls to get_type_hints > results in

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, 2021-04-13 at 11:33 +0900, Inada Naoki wrote: > On Tue, Apr 13, 2021 at 11:18 AM Paul Bryan wrote: > > > > On Tue, 2021-04-13 at 10:47 +0900, Inada Naoki wrote: > > > > On Tue, Apr 13, 2021 at 9:57 AM Larry Hastings > > wrote: > > > > > > This is really the heart of the debate over

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, Apr 13, 2021 at 11:18 AM Paul Bryan wrote: > > On Tue, 2021-04-13 at 10:47 +0900, Inada Naoki wrote: > > On Tue, Apr 13, 2021 at 9:57 AM Larry Hastings wrote: > > > This is really the heart of the debate over PEP 649 vs PEP 563. If you > examine an annotation, and it references an

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I've been thinking about this a bit, and I think that the way forward is for Python to ignore the text of annotations ("relaxed annotation syntax"), not to try and make it available as an expression. To be honest, the most pressing issue with annotations is the clumsy way that type variables have

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, 2021-04-13 at 10:47 +0900, Inada Naoki wrote: > On Tue, Apr 13, 2021 at 9:57 AM Larry Hastings > wrote: > > This is really the heart of the debate over PEP 649 vs PEP 563.  If > > you examine an annotation, and it references an undefined symbol, > > should that throw an error?  There is

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, Apr 13, 2021 at 9:57 AM Larry Hastings wrote: > > > On 4/12/21 4:50 PM, Inada Naoki wrote: > > PEP 563 solves all problems relating to types not accessible in runtime. > There are many reasons users can not get types used in annotations at runtime: > > * To avoid circular import > * Types

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/12/21 4:50 PM, Inada Naoki wrote: PEP 563 solves all problems relating to types not accessible in runtime. There are many reasons users can not get types used in annotations at runtime: * To avoid circular import * Types defined only in pyi files * Optional dependency that is slow to

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Tue, Apr 13, 2021 at 8:58 AM Larry Hastings wrote: > > On 4/12/21 4:50 PM, Inada Naoki wrote: > > As PEP 597 says, eval() is slow. But it can avoidable in many cases > with PEP 563 semantics. > > PEP 597 is "Add optional EncodingWarning". You said PEP 597 in one other > place too. Did you

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/12/21 4:50 PM, Inada Naoki wrote: As PEP 597 says, eval() is slow. But it can avoidable in many cases with PEP 563 semantics. PEP 597 is "Add optional EncodingWarning".  You said PEP 597 in one other place too.  Did you mean PEP 649 in both places? Cheers, //arry/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I still prefer PEP 563. I will describe what we lost if PEP 597 is accepted and PEP 563 is rejected. ### Types not accessible in runtime First of all, PEP 563 solves not only forward references. Note that PEP 563 says: "we'll call any name imported or defined within a `if TYPE_CHECKING: block`

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I'm a big fan of this PEP, for many reasons. But the fact that it addresses some of the issues with get_type_hints() is very important. dataclasses avoids calling get_type_hints() for performance reasons and because it doesn't always succeed, see https://github.com/python/typing/issues/508. I

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On Sun, 2021-04-11 at 23:34 -0700, Larry Hastings wrote: > Your example was valid, and I think your workaround should be fine.  > Do you have a use case for this, or is this question motivated purely > by curiosity? It took a few readings for me to understand the limitations in the PEP. My

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

On 4/11/21 7:55 PM, Paul Bryan wrote: PEP 563 also requires using ``eval()`` or ``typing.get_type_hints()`` to examine annotations. Code updated to work with PEP 563 that calls ``eval()`` directly would have to be updated simply to remove the ``eval()`` call. Code using

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

I like! I really appreciate the work you've put into this to get it this far. Questions and comments: > PEP 563 also requires using ``eval()`` or ``typing.get_type_hints()`` > to examine annotations. Code updated to work with PEP 563 that calls > ``eval()`` directly would have to be updated