[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Thomas Grainger 
I think as long as there's a test case for something like

```
@dataclass
class Node:
global_node: ClassVar[Node | None]
left: InitVar[Node | None]
right: InitVar[None | None]
```

the bug https://bugs.python.org/issue33453 and the current implementation 
https://github.com/python/cpython/blob/bfc2d5a5c4550ab3a2fadeb9459b4bd948ff61a2/Lib/dataclasses.py#L658-L714
 shows this is a tricky problem
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/FACVXQOACURN5PJROQLFADLCFCXUMP7K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Larry Hastings 

On 8/11/21 12:02 AM, Thomas Grainger wrote:

I think as long as there's a test case for something like

```
@dataclass
class Node:
 global_node: ClassVar[Node | None]
 left: InitVar[Node | None]
 right: InitVar[None | None]
```

the bug https://bugs.python.org/issue33453 and the current implementation 
https://github.com/python/cpython/blob/bfc2d5a5c4550ab3a2fadeb9459b4bd948ff61a2/Lib/dataclasses.py#L658-L714
 shows this is a tricky problem



The most straightforward workaround for this is to skip the decorator 
syntax.  With PEP 649 active, this code should work:


   class Node:
global_node: ClassVar[Node | None]
left: InitVar[Node | None]
right: InitVar[None | None]
   Node = dataclass(Node)


//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/ZWTTKKIOVA7UYKX4SFWJ6DZ5X72Z2O4P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Thomas Grainger 
Larry Hastings wrote:
> On 8/11/21 12:02 AM, Thomas Grainger wrote:
> > I think as long as there's a test case for something like
> > @dataclass
> > class Node:
> >  global_node: ClassVar[Node | None]
> >  left: InitVar[Node | None]
> >  right: InitVar[None | None]
> > 
> > the bug https://bugs.python.org/issue33453 and the current implementation 
> > https://github.com/python/cpython/blob/bfc2d5a5c4550ab3a2fadeb9459b4bd948ff6...
> >  shows this is a tricky problem
> > The most straightforward workaround for this is to skip the decorator 
> syntax.  With PEP 649 active, this code should work:
> class Node:
>  global_node: ClassVar[Node | None]
>  left: InitVar[Node | None]
>  right: InitVar[None | None]
> Node = dataclass(Node)
> //arry/

the decorator version simply has to work
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/2SHPHLZ2QV7YVCIC4PFPNLIHTW2WZP3A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Jukka Lehtosalo 
On Wed, Aug 11, 2021 at 10:32 AM Thomas Grainger  wrote:

> Larry Hastings wrote:
> > On 8/11/21 12:02 AM, Thomas Grainger wrote:
> > > I think as long as there's a test case for something like
> > > @dataclass
> > > class Node:
> > >  global_node: ClassVar[Node | None]
> > >  left: InitVar[Node | None]
> > >  right: InitVar[None | None]
> > >
> > > the bug https://bugs.python.org/issue33453 and the current
> implementation
> https://github.com/python/cpython/blob/bfc2d5a5c4550ab3a2fadeb9459b4bd948ff6...
> shows this is a tricky problem
> > > The most straightforward workaround for this is to skip the decorator
> > syntax.  With PEP 649 active, this code should work:
> > class Node:
> >  global_node: ClassVar[Node | None]
> >  left: InitVar[Node | None]
> >  right: InitVar[None | None]
> > Node = dataclass(Node)
> > //arry/
>
> the decorator version simply has to work
>

I also think that it would be unfortunate if the decorator version wouldn't
work. This is a pretty basic use case. If we go through a lot of trouble to
redesign how type annotations behave, it would be great if we could address
as many common pain points as possible. Otherwise I see a risk that we'll
have another (third!) redesign in Python 3.12 or 3.13.

Jukka
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/6U27SVAIHIRTFFJVKJWCNIXEJM6YFO6H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Paul Moore 
On Wed, 11 Aug 2021 at 07:37, Larry Hastings  wrote:
>
[...]
> I think PEP 649 should be considered in the same way.  In my opinion, the 
> important thing is to figure out what semantics we want for the language.

I think this is a very important point. As originally implemented,
annotations allowed attaching arbitrary data to function arguments and
return values. That was a simple language capability, with no
particular nuances, although the intended use case was very much type
annotations. Later, the decision was made that typing would be the
primary and "official" use for annotations - but even at that point,
the semantics were still impartial.

String annotations are changing the fundamental semantics of
annotations (introspection returns the source code string, not a
Python object) and there's even discussion of how this allows the
possibility for annotations to not even conform to standard Python
syntax. Surely, before we start changing the fundamental syntax and
semantics of a language feature for a single (albeit officially
blessed) use case, we should look at what capabilities *in the
abstract* are missing from the current behaviour, and work out how or
if we can implement them. Proposers on python-ideas are routinely
asked to demonstrate how their suggestion affects cases other than
their particular use case. Why shouldn't we hold typing to a similar

PEP 563 states that it's addressing two issues:

* Forward references for type hints
* Module import time cost of having type hints at all

Both of those are real and valid problems that deserve a solution.

PEP 563 is asserting that no solution is possible within the existing
semantics, and it's necessary to significantly break backward
compatibility in terms of semantics, and also opens up the possibility
of annotations not even being normal Python syntax in future (the PEP
says that any such change would need a further PEP, but definitely
does *not* say that the possibility is out of the question). PEP 649,
in my opinion, is exploring the option of addressing these two issues
*within Python's existing syntax and semantics*. How we look at it IMO
depends significantly on how we view the whole question of typing and
what it means for typing to be "optional".

Personally, my experience is that typing is indeed optional for small
or personal projects. But for larger projects, where there's a team of
people involved, it's very hard to argue against typing. Even if
you're not convinced personally that the benefits justify the costs,
it's hard to make that case persuasive, so typing pretty much always
wins.

With that in mind, if we *are* saying (as PEP 563 is, in effect) that
we're at the breaking point where we cannot support the typing use
case with existing Python syntax (note that PEP 563 essentially makes
annotations a new syntax for string literals[1]) then I would far
prefer it if Python gave up on this ambivalent attitude towards
annotations, and made a proper commitment to having a way of
expressing types within the language. That means defining type syntax
as Python *language* syntax and treating type syntax as subject to all
of the compatibility guarantees that the rest of the language adheres
to. It may still mean that we define new, type-specific syntax, but we
can put it in the language definition, and say that it's only valid in
annotations. That's not a showstopper. And it means that we don't open
ourselves up to tool-specific interpretation of core Python syntax.

What that would mean is:

1. What is valid as an annotation would probably need to be specified
once and for all at the language level. That's a compatibility break,
but we'd be making it with our eyes open, having found that the "a
single Python expression" approach isn't sufficient for the typing use
case.
2. The forward reference issue is treated as a *python language*
issue, not just a typing issue. It may be that we decide to only
*solve* it for typing, but we can still look to the wider context for
ideas and inspiration.
3. The idea of having "special syntax" for type expressions would be
dropped - it's not "special syntax" if it's part of the language, any
more than assignment expressions or conditional expressions are
"special syntax".

It's possible that this is something that's already been discussed in
the typing community. But if so, then that's a discussion that should
have involved python-dev, as it's fairly basic to how Python as a
language is evolving.

On the other hand, PEP 647 is saying that we think that typing can
still be supported with existing syntax, and we don't need to (yet)
have that discussion. Although maybe the typing experts have further
plans (the whole "support non-Python syntax" hint suggests that they
might) that mean new syntax is an inevitability - but if so, let's get
that proposal out in the open where it can be discussed, and not use
PEP 563 as a "back door" to avoid that discussion.

[...]
>  Once we figure out what semantics we want

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Larry Hastings 


On 8/11/21 2:48 AM, Jukka Lehtosalo wrote:
On Wed, Aug 11, 2021 at 10:32 AM Thomas Grainger > wrote:


Larry Hastings wrote:
> On 8/11/21 12:02 AM, Thomas Grainger wrote:
> > I think as long as there's a test case for something like
> > @dataclass
> > class Node:
> >      global_node: ClassVar[Node | None]
> >      left: InitVar[Node | None]
> >      right: InitVar[None | None]
> >
> > the bug https://bugs.python.org/issue33453
 and the current
implementation

https://github.com/python/cpython/blob/bfc2d5a5c4550ab3a2fadeb9459b4bd948ff6.

..
shows this is a tricky problem
> > The most straightforward workaround for this is to skip the
decorator
> syntax.  With PEP 649 active, this code should work:
> class Node:
>          global_node: ClassVar[Node | None]
>          left: InitVar[Node | None]
>          right: InitVar[None | None]
>     Node = dataclass(Node)
> //arry/

the decorator version simply has to work


I also think that it would be unfortunate if the decorator version 
wouldn't work. This is a pretty basic use case.



So, here's an idea, credit goes to Eric V. Smith.  What if we tweak how 
decorators work, /jst slghtly/, so that they work like the 
workaround code above?


Specifically: currently, decorators are called just after the function 
or class object is created, before it's bound to a variable.  But we 
could change it so that we first bind the variable to the initial value, 
then call the decorator, then rebind.  That is, this code:


   @dekor8
   class C:
    ...

would become equivalent to this code:

   class C:
    ...
   C = dekorate(C)

This seems like it would solve the class self-reference problem--the 
"Node" example above--when PEP 649 is active.


This approach shouldn't break reasonable existing code.  That said, this 
change would be observable from Python, and pathological code could 
notice and break.  For example:


   def ensure_Foo_is_a_class(o):
    assert isinstance(Foo, type)
    return o

   class Foo:
    ...

   @ensure_Foo_is_a_class
   def Foo():
    ...

This terrible code currently would not raise an assertion.  But if we 
made the proposed change to the implementation of decorators, it would.  
I doubt anybody does this sort of nonsense, I just wanted to fully flesh 
out the topic.



If this approach seems interesting, here's one wrinkle to iron out.  
When an object has multiple decorators, would we want to re-bind after 
each decorator call?  That is, would


   @dekor1
   @dekor2
   @dekor3
   class C:
    ...

turn into approach A:

   class C:
    ...
   C = dekor1(dekor2(dekor3(C)))

or approach B:

   class C:
    ...
   C = dekor3(C)
   C = dekor2(C)
   C = dekor1(C)

I definitely think "approach B" makes more sense.


//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/OA4J23TU3XACE5NAUUQNQQ52BXGNHUIS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Chris Angelico 
On Wed, Aug 11, 2021 at 10:03 PM Larry Hastings  wrote:
> Specifically: currently, decorators are called just after the function or 
> class object is created, before it's bound to a variable.  But we could 
> change it so that we first bind the variable to the initial value, then call 
> the decorator, then rebind.  That is, this code:
>
> @dekor8
> class C:
> ...
>
> would become equivalent to this code:
>
> class C:
> ...
> C = dekorate(C)
>
> This seems like it would solve the class self-reference problem--the "Node" 
> example above--when PEP 649 is active.

Critically, the decorator itself would have to be evaluated *before*
that assignment. So it would actually have to work out more like:

_tmp = dekor8
class C:
...
C = _tmp(C)

Otherwise things like "@foo.setter" would break.

> This approach shouldn't break reasonable existing code.  That said, this 
> change would be observable from Python, and pathological code could notice 
> and break.  For example:
>
> def ensure_Foo_is_a_class(o):
> assert isinstance(Foo, type)
> return o
>
> class Foo:
> ...
>
> @ensure_Foo_is_a_class
> def Foo():
> ...
>
> This terrible code currently would not raise an assertion.  But if we made 
> the proposed change to the implementation of decorators, it would.  I doubt 
> anybody does this sort of nonsense, I just wanted to fully flesh out the 
> topic.
>

You would be here declaring that a @monkeypatch decorator is terrible
code. I'm not sure whether you're right or wrong. You may very well be
right.

def monkeypatch(cls):
basis = globals()[cls.__name__]
for attr in dir(cls): setattr(basis, attr, getattr(cls, attr))
return basis

@monkeypatch
class SomeClass:
def new_method(self): ...

Currently this works, since SomeClass doesn't get assigned yet. This
could be made to work across versions by writing it as
@monkeypatch(SomeClass) instead (and then the actual class name would
become immaterial).

> If this approach seems interesting, here's one wrinkle to iron out.  When an 
> object has multiple decorators, would we want to re-bind after each decorator 
> call?  That is, would
>
> @dekor1
> @dekor2
> @dekor3
> class C:
> ...
>
> turn into approach A:
>
> class C:
> ...
> C = dekor1(dekor2(dekor3(C)))
>
> or approach B:
>
> class C:
> ...
> C = dekor3(C)
> C = dekor2(C)
> C = dekor1(C)
>
> I definitely think "approach B" makes more sense.
>

If you're going to do it at all, best go the whole way. Each decorator
functions independently, and a decorated class is a class like any
other, so approach B makes far more sense to me.

ChrisA
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/KSXPPKYUPWYOGW2MROMQJRXPWBEH5BIW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Inada Naoki 

> 2021/08/11 19:22、Paul Moore のメール:
> 
> Also, I don't think that improving performance is a
> justification for a non-trivial backward compatibility break (I don't
> recall a case where we've taken that view in the past) so "PEP 649
> solves forward references without a backward compatibility impact, and
> performance is a small issue in the face of that" is a reasonable
> position to take.

OK. I will stop talking about import time.

But memory footprint and GC time is still an issue.
Annotations in PEP 649 semantics can be much heavier than docstrings.

So I want to measure them before deprecating 563.

IMHO, I don't think we can make PEP 563 default anymore.
FastAPI ecosystem changed how annotations are used after PEP 563 was accepted.

So I think two or three options are on the table:

a. Keep Python 3.10 state. PEP 563 is opt-in in foreseeable future.
b. Accept PEP 649 and deprecate both of old semantics and PEP 563.
c?. Accept PEP 649 and deprecate old semantics, but keep PEP 563 as opt-in.

(I exclude "Accept PEP 649 but keep old semantics" option because backward 
compatibility is not a problem if we keep old semantics.)

Of course, (b) is the most simple solution.
But I am not sure that (b) don't have significant memory usage and GC time 
regression.

We can keep (a) in Python 3.11 so we don't have to harry about accepting PEP 
649.

Regards,

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/QTAXVWFNF632TPCCLNFJTAXPRMV45SO6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Larry Hastings 

On 8/11/21 5:21 AM, Inada Naoki wrote:

But memory footprint and GC time is still an issue.
Annotations in PEP 649 semantics can be much heavier than docstrings.



I'm convinced that, if we accept PEP 649 (or something like it), we can 
reduce its CPU and memory consumption.


Here's a slightly crazy idea I had this morning: what if we didn't 
unmarshall the code object for co_annotation during the initial import, 
but instead lazily loaded it on demand?  The annotated object would 
retain knowledge of what .pyc file to load, and what offset the 
co_annotation code object was stored at. (And, if the co_annotations 
function had to be a closure, a reference to the closure tuple.)  If the 
user requested __annotations__ (or __co_annotations__), the code would 
open the .pyc file, unmarshall it, bind it, etc.  Obviously this would 
only work for code loaded from .pyc (etc) files.  To go even crazier, 
the runtime could LRU cache N (maybe == 1) open .pyc file handles as a 
speed optimization, perhaps closing them after some wall-clock timeout.


I doubt we'd do exactly this--it's easy to find problems with the 
approach.  But maybe this idea will lead to a better one?



//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/JMLXOEV6GRBVPNHQD6CYEW63I7WJBWZC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Thomas Grainger 
Larry Hastings wrote:
> On 8/11/21 2:48 AM, Jukka Lehtosalo wrote:
> > On Wed, Aug 11, 2021 at 10:32 AM Thomas Grainger  > mailto:[email protected]> wrote:
> > Larry Hastings wrote:
> > > On 8/11/21 12:02 AM, Thomas Grainger wrote:
> > > > I think as long as there's a test case for something like
> > > > @dataclass
> > > > class Node:
> > > >      global_node: ClassVar[Node | None]
> > > >      left: InitVar[Node | None]
> > > >      right: InitVar[None | None]
> > > >
> > > > the bug https://bugs.python.org/issue33453
> >  and the current
> > implementation
> > https://github.com/python/cpython/blob/bfc2d5a5c4550ab3a2fadeb9459b4bd948ff6.
> > ..
> > shows this is a tricky problem
> > > > The most straightforward workaround for this is to skip the
> > decorator
> > > syntax.  With PEP 649 active, this code should work:
> > > class Node:
> > >          global_node: ClassVar[Node | None]
> > >          left: InitVar[Node | None]
> > >          right: InitVar[None | None]
> > >     Node = dataclass(Node)
> > > //arry/
> > 
> > the decorator version simply has to work
> > 
> > 
> > 
> > I also think that it would be unfortunate if the decorator version 
> > wouldn't work. This is a pretty basic use case.
> > So, here's an idea, credit goes to Eric V. Smith.  What if we tweak how 
> decorators work, /jst slghtly/, so that they work like the 
> workaround code above?
> Specifically: currently, decorators are called just after the function 
> or class object is created, before it's bound to a variable.  But we 
> could change it so that we first bind the variable to the initial value, 
> then call the decorator, then rebind.  That is, this code:
> @dekor8
> class C:
>      ...
> would become equivalent to this code:
> class C:
>      ...
> C = dekorate(C)
> This seems like it would solve the class self-reference problem--the 
> "Node" example above--when PEP 649 is active.
> This approach shouldn't break reasonable existing code.  That said, this 
> change would be observable from Python, and pathological code could 
> notice and break.  For example:
> def ensure_Foo_is_a_class(o):
>      assert isinstance(Foo, type)
>      return o
> class Foo:
>      ...
> @ensure_Foo_is_a_class
> def Foo():
>      ...
> This terrible code currently would not raise an assertion.  But if we 
> made the proposed change to the implementation of decorators, it would.  
> I doubt anybody does this sort of nonsense, I just wanted to fully flesh 
> out the topic.
> If this approach seems interesting, here's one wrinkle to iron out.  
> When an object has multiple decorators, would we want to re-bind after 
> each decorator call?  That is, would
> @dekor1
> @dekor2
> @dekor3
> class C:
>      ...
> turn into approach A:
> class C:
>      ...
> C = dekor1(dekor2(dekor3(C)))
> or approach B:
> class C:
>      ...
> C = dekor3(C)
> C = dekor2(C)
> C = dekor1(C)
> I definitely think "approach B" makes more sense.
> //arry/


You mention that you wanted this to work also for non-type hint usage of 
annotations, and so a ForwardRef won't work. As such, would you also change 
this for function decorators so you can do this?

```
@decorator
@typing.no_type_check
def ham(spam: ham):
...

So it means:

```
def ham(spam: ham):
...
ham = typing.no_type_check(ham)
ham = decorator(ham)
```

Obviously it's meaningless as a type hint, hence the no_type_check opt out
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/OJKTIGCHNFTW224O6OM4NIZ3OZBK2YQI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Guido van Rossum 
As it happens, I have a working prototype of lazy in marshaling that would
work well for this.

On Wed, Aug 11, 2021 at 06:07 Larry Hastings  wrote:

> On 8/11/21 5:21 AM, Inada Naoki wrote:
>
> But memory footprint and GC time is still an issue.
> Annotations in PEP 649 semantics can be much heavier than docstrings.
>
>
> I'm convinced that, if we accept PEP 649 (or something like it), we can
> reduce its CPU and memory consumption.
>
> Here's a slightly crazy idea I had this morning: what if we didn't
> unmarshall the code object for co_annotation during the initial import, but
> instead lazily loaded it on demand?  The annotated object would retain
> knowledge of what .pyc file to load, and what offset the co_annotation code
> object was stored at.  (And, if the co_annotations function had to be a
> closure, a reference to the closure tuple.)  If the user requested
> __annotations__ (or __co_annotations__), the code would open the .pyc file,
> unmarshall it, bind it, etc.  Obviously this would only work for code
> loaded from .pyc (etc) files.  To go even crazier, the runtime could LRU
> cache N (maybe == 1) open .pyc file handles as a speed optimization,
> perhaps closing them after some wall-clock timeout.
>
> I doubt we'd do exactly this--it's easy to find problems with the
> approach.  But maybe this idea will lead to a better one?
>
>
> */arry*
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/JMLXOEV6GRBVPNHQD6CYEW63I7WJBWZC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/6I4WN5IAPXLIP4JHGWIJ6MIOFUZQU62E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Guido van Rossum 
Oh, I agree it shouldn’t reference the typing module. But it should not
raise NameError. This whole thing already is a special case. We can debate
what else it should, e.g. skip the name, return a fixed error token, return
an error token that includes the name that failed (this is part if the
NameError), return a string, etc…

FWIW the “tiny” change to decorator syntax looks untenable.

On Tue, Aug 10, 2021 at 23:09 Larry Hastings  wrote:

> On 8/10/21 11:15 AM, Thomas Grainger wrote:
>
> Although the co_annoations code could intercept the NameError and replace 
> return a ForwardRef object instead of the resolved name
>
>
> No, it should raise a NameError, just like any other Python code.
> Annotations aren't special enough to break the rules.
>
> I worry about Python-the-language enshrining design choices made by the
> typing module.  Python is now on its fourth string interpolation
> technology, and it ships with three command-line argument parsing
> libraries; in each of these cases, we were adding a New Thing that was
> viewed at the time as an improvement over the existing thing(s).  It'd be
> an act of hubris to assert that the current "typing" module is the
> ultimate, final library for expressing type information in Python.  But if
> we tie the language too strongly to the typing module, I fear we could
> strangle its successors in their cribs.
>
>
> */arry*
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/5NUVYOLIHEHS373WEK3D2ZGXSWPI5XV7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/CJRLXDSPFEP3V2PN2YIGQHXTQ6A262ZN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Thomas Grainger 
Would:
```
@dataclass
class Node:
global_node: __class__ | None
```

"Just work" with co_annotations?
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/ONQM6PTLD5QFC6UAJIYH437XFMAEEOAD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Jukka Lehtosalo 
On Wed, Aug 11, 2021 at 2:56 PM Thomas Grainger  wrote:

> Would:
> ```
> @dataclass
> class Node:
> global_node: __class__ | None
> ```
>
> "Just work" with co_annotations?
>

This feels too specialized to me.

It would be great to also handle forward references to other classes and
cyclic references, which are also somewhat common:

@dataclass
class NodeA:
component: NodeB | None

@dataclass
class NodeB:
component: NodeA | None

Another, slightly more complex example would be cyclic references within
two modules in an import cycle. For example, NodeA and NodeB could be
defined in different modules. The common thing is that the dependency cycle
can only be fully resolved after we have created both type objects. The
import cycle case is probably less common but I've seen it in real-world
code.

Jukka
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/XFS6VSXGV6QMEY5AB7VO64OZMHAYSDSZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Steve Holden 
On Wed, Aug 11, 2021 at 2:27 PM Guido van Rossum  wrote:

> As it happens, I have a working prototype of lazy in marshaling that would
> work well for this.
>
> Nice to see you keep the time machine in working order ...

Kind regards,
Steve
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/PIMCZCDZ2SN2QDAFE3ORZTURIURHYF4K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Steve Holden 
On Wed, Aug 11, 2021 at 7:09 AM Larry Hastings  wrote:

> On 8/10/21 11:15 AM, Thomas Grainger wrote:
>
> Although the co_annoations code could intercept the NameError and replace 
> return a ForwardRef object instead of the resolved name
>
>
> No, it should raise a NameError, just like any other Python code.
> Annotations aren't special enough to break the rules.
>
> I worry about Python-the-language enshrining design choices made by the
> typing module.  Python is now on its fourth string interpolation
> technology, and it ships with three command-line argument parsing
> libraries; in each of these cases, we were adding a New Thing that was
> viewed at the time as an improvement over the existing thing(s).  It'd be
> an act of hubris to assert that the current "typing" module is the
> ultimate, final library for expressing type information in Python.  But if
> we tie the language too strongly to the typing module, I fear we could
> strangle its successors in their cribs.
>
>
> Which would be unfortunate given the (explicit?) assurances that
annotations would be optional; they are casting their shadow over the whole
language.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/XFR3KNKDYYXDXVQNDP6XPTGBBHA32AVI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Terry Reedy 

On 8/11/2021 7:56 AM, Larry Hastings wrote:

So, here's an idea, credit goes to Eric V. Smith.  What if we tweak how 
decorators work, /jst slghtly/, so that they work like the 
workaround code above?


Specifically: currently, decorators are called just after the function 
or class object is created, before it's bound to a variable.  But we 
could change it so that we first bind the variable to the initial value, 
then call the decorator, then rebind.  That is, this code:


@dekor8
class C:
     ...

would become equivalent to this code:

class C:
     ...
C = dekorate(C)


This is how function decorators were originally defined. Before the 2016 
(3.5) revision of

https://docs.python.org/3/reference/compound_stmts.html#function-definitions
by https://bugs.python.org/issue26576

---
   @f1(arg)
   @f2
   def func(): pass

is equivalent to

   def func(): pass
   func = f1(arg)(f2(func))
---

After
---

@f1(arg)
@f2
def func(): pass

is roughly equivalent to

def func(): pass
func = f1(arg)(f2(func))

except that the original function is not temporarily bound to the name func.
---

I questioned on the issue whether the non-binding optimization "should 
it be documented as a guaranteed language feature or as just an optional 
optimization?"


--
Terry Jan Reedy


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/PG4UIVT3XEWUS5TVO4MJIQV47UOVV3TD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Larry Hastings 

On 8/11/21 5:15 AM, Chris Angelico wrote:

On Wed, Aug 11, 2021 at 10:03 PM Larry Hastings  wrote:

This approach shouldn't break reasonable existing code.  That said, this change 
would be observable from Python, and pathological code could notice and break.  
For example:

def ensure_Foo_is_a_class(o):
 assert isinstance(Foo, type)
 return o

class Foo:
 ...

@ensure_Foo_is_a_class
def Foo():
 ...

This terrible code currently would not raise an assertion.  But if we made the 
proposed change to the implementation of decorators, it would.  I doubt anybody 
does this sort of nonsense, I just wanted to fully flesh out the topic.


You would be here declaring that a @monkeypatch decorator is terrible
code. I'm not sure whether you're right or wrong. You may very well be
right.

def monkeypatch(cls):
 basis = globals()[cls.__name__]
 for attr in dir(cls): setattr(basis, attr, getattr(cls, attr))
 return basis

@monkeypatch
class SomeClass:
 def new_method(self): ...

Currently this works, since SomeClass doesn't get assigned yet. This
could be made to work across versions by writing it as
@monkeypatch(SomeClass) instead (and then the actual class name would
become immaterial).



Golly!  I've never seen that.  Is that a common technique?

If we need to preserve that behavior, then this idea is probably a 
non-starter.



//arry/

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/QNHESUROGJ7V4Q5LFUNXCU2FUWNOW7OE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Thomas Grainger 
I don't think I've seen code like this. It would be incredibly useful to have a 
cpython-primer (like mypy-primer and black-primer) that ran as many test suites 
as possible from pypi with every cPython commit
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/QEGUZ5GOKA4CI5ELA5BBDSS76UJN26BI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Barry Warsaw 
On Aug 9, 2021, at 08:27, Eric V. Smith  wrote:
> 
> My understanding is that PEP 649 is currently in front of the SC. But do we 
> need to have any additional discussion here? My recollection is that we 
> backed out the PEP 563 change because we didn't feel we had enough time to 
> come to a good decision one way or the other before 3.10.

My recollection for why the SC deferred this (other than running out of time 
and not wanting to make a hasty decision for 3.10) is two fold:

* Should the type annotation syntax always follow Python syntax, or should we 
allow the type annotation syntax to deviate from “standard” Python?  This also 
came up in the context of PEP 646, which introduces syntax for type annotations 
that may or may not be useful or desired for regular Python.

* Should we codify current practice so that type annotations are useful and 
supported at both compile time and run time (e.g. the pydantic use case)?

These are still open questions.  While it’s interesting to entertain the 
separation of general Python syntax from type annotation syntax — allowing the 
latter to evolve more quickly and independently from the former -- ultimately I 
think (and my gauge of the SC sentiment is) that there’s no way this will work 
in practice.  That means that type annotation syntax can’t be delegated and so 
we have to consider type-inspired syntax changes within the full context of the 
Python language.

As for the second question, it means that we have to be very careful that the 
folks who use type annotation at compile/static checking time (e.g. mypy and 
friends) explicitly consider the existing use cases and needs of the runtime 
type community.  These two constituents have to work together to avoid backward 
incompatible changes.

When the SC was considering all these PEPs for 3.10, we felt like we needed 
clarity on the direction and purpose of type annotations before we could make 
decisions we’d have to live with essentially forever.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/HWBE6DXFAF7LAXX3FEG2PFA4BW3DHE5S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Eric V. Smith 

On 8/11/2021 11:07 AM, Jukka Lehtosalo wrote:
On Wed, Aug 11, 2021 at 2:56 PM Thomas Grainger > wrote:


Would:
```
@dataclass
class Node:
    global_node: __class__ | None
```

"Just work" with co_annotations?


This feels too specialized to me.

It would be great to also handle forward references to other classes 
and cyclic references, which are also somewhat common:


@dataclass
class NodeA:
component: NodeB | None

@dataclass
class NodeB:
component: NodeA | None


Note that for dataclasses itself, you can just use:

@dataclass
class NodeA:
component: "NodeB | None"

@dataclass
class NodeB:
component: "NodeA | None"

It's only looking for typing.ClassVar or dataclasses.InitVar. Anything 
else, including a string, just means "this is a normal field". This will 
result in dataclasses.fields(NodeA)[0].type being the string "NodeB | 
None". But that's okay with dataclasses.


However, there are a number of open bpo issues where people want 
dataclasses to resolve the .type value to be the actual type object 
instead of a string, especially if using 'from __future__ import 
annotations'. I guess dataclasses.fields() could assist there, optionally.


Eric



Another, slightly more complex example would be cyclic references 
within two modules in an import cycle. For example, NodeA and NodeB 
could be defined in different modules. The common thing is that the 
dependency cycle can only be fully resolved after we have created both 
type objects. The import cycle case is probably less common but I've 
seen it in real-world code.


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/7GU6PYY2JDAY52FQ3ZXYJN3RDZS4RN5N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Should we try to get a complete draft of What's New by b1?

2021-08-11 Thread Brett Cannon 
Not to specifically pick on Eric V. Smith, but yesterday a co-worker came
to me asking about `KW_ONLY` for dataclasses and wanting to provide
feedback (
https://docs.python.org/3.10/library/dataclasses.html#dataclasses.KW_ONLY).
I said we had reached RC and so it was pretty much too late; the alphas and
betas are when one should provide feedback.

But then I realized you wouldn't even know about this new feature unless
you read What's New ... and that's when I found out this new feature isn't
mentioned (yet). As of right now the only dataclass feature documented in
the What's New is something about slots (
https://docs.python.org/3.10/whatsnew/3.10.html#dataclasses).

So my question is whether we want to push to be more diligent about
updating What's New by b1 so people can provide feedback during the betas
beyond just reporting breaking changes?
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/YTXDZRAVW3VNVMJUCFZYRWNECZFKEFDH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we try to get a complete draft of What's New by b1?

2021-08-11 Thread Eric V. Smith 
Yes, I think we should get it done by b1. For some reason it never 
occurred to me to add this to What's New. Someone just reported this on 
bpo this week. I'll try and get a PR together this week.


Eric (who doesn't feel picked on!)

On 8/11/2021 2:35 PM, Brett Cannon wrote:
Not to specifically pick on Eric V. Smith, but yesterday a co-worker 
came to me asking about `KW_ONLY` for dataclasses and wanting to 
provide feedback 
(https://docs.python.org/3.10/library/dataclasses.html#dataclasses.KW_ONLY 
). 
I said we had reached RC and so it was pretty much too late; the 
alphas and betas are when one should provide feedback.


But then I realized you wouldn't even know about this new feature 
unless you read What's New ... and that's when I found out this new 
feature isn't mentioned (yet). As of right now the only dataclass 
feature documented in the What's New is something about slots 
(https://docs.python.org/3.10/whatsnew/3.10.html#dataclasses 
).


So my question is whether we want to push to be more diligent about 
updating What's New by b1 so people can provide feedback during the 
betas beyond just reporting breaking changes?


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/YTXDZRAVW3VNVMJUCFZYRWNECZFKEFDH/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/EKAP5BVOFKNAAOW3ZV5PMDELFAJJO6QY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Luciano Ramalho 
On Wed, Aug 11, 2021 at 2:13 PM Barry Warsaw  wrote:
> As for the second question, it means that we have to be very careful that the 
> folks who use type annotation at compile/static checking time (e.g. mypy and 
> friends) explicitly consider the existing use cases and needs of the runtime 
> type community.  These two constituents have to work together to avoid 
> backward incompatible changes.

Very well put, thank you Barry.

As I see it, the companies who did most of the investment in typing
want the benefits static typing (as most of us do), but widespread use
of type hints in large codebases created another problem: static
checking only happens in developer workstations and CI servers, but
*imports* happen all the time in code running in production, on
countless servers. That became a real issue for those very same
companies operating at Web scale.

So they have a strong incentive to focus on the use of annotations for
static checking only, while many of us also want type hints to address
use cases where Python is used as a *dynamic* language, which is its
nature, and likely a strong reason for its popularity in the first
place—despite the inherent runtime costs of being a dynamic language.

Cheers,

Luciano


-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
|  Technical Principal at ThoughtWorks
|  Twitter: @ramalhoorg
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/DAC3OOMPQ3SOB6CKVEHHJMJHLT6FNICY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Chris Angelico 
On Thu, Aug 12, 2021 at 3:01 AM Larry Hastings  wrote:
>
> On 8/11/21 5:15 AM, Chris Angelico wrote:
>
> On Wed, Aug 11, 2021 at 10:03 PM Larry Hastings  wrote:
>
> This approach shouldn't break reasonable existing code.  That said, this 
> change would be observable from Python, and pathological code could notice 
> and break.  For example:
>
> def ensure_Foo_is_a_class(o):
> assert isinstance(Foo, type)
> return o
>
> class Foo:
> ...
>
> @ensure_Foo_is_a_class
> def Foo():
> ...
>
> This terrible code currently would not raise an assertion.  But if we made 
> the proposed change to the implementation of decorators, it would.  I doubt 
> anybody does this sort of nonsense, I just wanted to fully flesh out the 
> topic.
>
> You would be here declaring that a @monkeypatch decorator is terrible
> code. I'm not sure whether you're right or wrong. You may very well be
> right.
>
> def monkeypatch(cls):
> basis = globals()[cls.__name__]
> for attr in dir(cls): setattr(basis, attr, getattr(cls, attr))
> return basis
>
> @monkeypatch
> class SomeClass:
> def new_method(self): ...
>
> Currently this works, since SomeClass doesn't get assigned yet. This
> could be made to work across versions by writing it as
> @monkeypatch(SomeClass) instead (and then the actual class name would
> become immaterial).
>
>
> Golly!  I've never seen that.  Is that a common technique?
>
> If we need to preserve that behavior, then this idea is probably a 
> non-starter.
>

This specific thing? No, it's not common. But it's a natural
consequence of the current behaviour, so if it does change, there'll
be a variety of things that will break. It's hard to know how much of
that is good code and how much is bad code.

I have frequently made decorators that do unusual things (like
returning non-functions) and I'm sure that some of them would be
considered abuse of functionality :)

ChrisA
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/AJZEUIQTBOFS74CYAZ2R5WXBWRLDPF4X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Problems with dict subclassing performance

2021-08-11 Thread Marco Sulla 
No ideas? Excuse me for the up.

On Fri, 6 Aug 2021 at 21:29, Marco Sulla  wrote:
>
> I've done an answer on SO about why subclassing `dict` makes the
> subclass so much slower than `dict`. The answer is interesting:
>
> https://stackoverflow.com/questions/59912147/why-does-subclassing-in-python-slow-things-down-so-much
>
> What do you think about?
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/ODHRCDEPOP3AUJJ6IKF2GFLOWJ5BY7TX/
Code of Conduct: http://python.org/psf/codeofconduct/