[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

And I just had a realization.  Lazy creation of an empty annotations dict, 
for both classes and modules, will work fine.

As stated in my previous comment in this issue, my goal here is to improve best 
practices in 3.10+, while preserving the unfortunate best practices of 3.9 and 
before.  I don't know why I couldn't see it before, but: adding a getset to 
modules and user classes, and lazily creating the empty annotations dict if not 
set, ought to work fine.  The getset code will have to store the annotations 
dict in the class / module dict but that's not a big deal.

If you're in 3.10+, and your code looks in the class dict for annotations, and 
we lazy-create it with a getset and store it in the class dict, then you still 
see what you expected to see.

* If the class has annotations, they'll be in the class dict.
* If the class has no annotations, but someone accessed
  cls.__annotations__, the empty dict will be lazily created
  and you'll see it.
* If the class doesn't have annotations, then the class dict
  won't have an '__annotations__' key, which is what you were expecting.

In any scenario the old best practices code works fine.

If you're in 3.10+, and your code uses cls.__annotations__ or getattr() to get 
the class dict, and we lazy-create it with a getset and store it in the class 
dict, then you also get what you expected.  If the class or module has 
annotations, you get them; if it doesn't, it lazy creates an empty dict and 
stores and returns that.  Your code works fine too.

Sorry I've been slow on figuring this out--but at least I got there in the end, 
before beta.  And it's good news!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

I'm please you folks are as supportive as you are of what I'm doing here, given 
that you seem a little unsure of the details.  I concede that there's a lot 
going on and it can be hard to keep it all in your head.

The point of this issue / PR is to improve the best practices for 3.10 without 
breaking the best practices for 3.9 and before.

Best practice in 3.10 is to use inspect.get_annotations(), and failing that, 
three-argument getattr(), for all annotated objects.  This issue / PR permits 
that to happen.

Best practices for 3.9 differed between different objects.  For classes, you 
had no choice but to look in the class dict because of the inheritance problem. 
 (Either that, or, the elaborate scheme 'attrs' used.)  That code will 
emphatically _still work_ in 3.10.  We are not breaking backwards compatibility.

So, the best practices in 3.9 and before will still work in 3.10.  But the best 
practices for 3.10+ are improved because of the work we're doing here and in 
other places.

My intent was to document the best practices for 3.10+.  If folks think it 
would be helpful, this same section in the 3.10 docs can also describe best 
practices for 3.9 and before.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Eric V. Smith]

Eric V. Smith  added the comment:

"It's fine to have advice ..."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Eric V. Smith]

Eric V. Smith  added the comment:

It's find to have advice of "do X in 3.9", and "do Y in 3.10+". I think the 
issue is: if you have code that needs to run in 3.9 and 3.10+, what should you 
do? There needs to be some advice for that case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

I'm not breaking backwards compatibility--that's the point of all this.  But 
I'm improving the experience.  And if you don't care about 3.9 and before, you 
can stick to the new improved experience.

Looking in the class dict for annotations is terrible, but that's best practice 
for 3.9 and earlier because of the flawed design of annotations.

For 3.10+, best practice to look at annotations is inspect.get_annotations() 
for all objects.  But this isn't best practice for 3.9, because the function 
didn't exist in 3.9.  Also, the paragraph might go on to say, if you have a 
good reason why you can't call inspect.get_annotations(), best practice no 
longer requires you to look in the class dict, because of this very issue/PR 
that is behavior we are changing for 3.10.

So, best practices changed in 3.10.  And I plan to add a section to the docs 
somewhere that says "Here are best practices for accessing annotations in 
3.10+".  As I said, if you think the Python 3.10 docs should have a section 
about "here are best practices in 3.9", I can add that too, but it'll be a 
different paragraph.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Guido van Rossum]

Guido van Rossum  added the comment:

And that’s a problem. There needs to be a recommendation on what to do for code 
that spans 3.9 and 3.10. What should users do otherwise? Drop 3.9 as soon as 
they introduce 3.10 support? Withhold 3.10 support until 3.9 reaches EOL?

IOW you can’t just break backward compatibility. (That’s why PEP 563 was rolled 
back in the first place.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

> I’d say that best practices for 3.9+ are more useful.

My point in writing this up was that the best practices change as of 3.10.  So, 
I could add a section to the Python 3.10 documentation giving best practices 
for 3.10+ and 3.9-.  But 3.9 and 3.10 have different best practices.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Guido van Rossum]

Guido van Rossum  added the comment:

I’d say that best practices for 3.9+ are more useful.

There seem to be several contradictions in your remarks, but we’ll get to those 
in the review.

--
nosy: +Guido.van.Rossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

> * Never modify o.__annotations__.

Let me revise that to:

* If o.__annotations__ is a dict, never modify the contents of that dict.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

It occurs to me that part of this work should also be a new "best practices for 
__annotations__" entry in the Python docs.


Best practices for working with annotations, for code that requires a minimum 
Python version of 3.10+:

Best practice is to call either inspect.get_annotations() or 
typing.get_type_hints() to access annotations.  But if you want to access 
'__annotations__' on an object directly:

* Always access the annotations on an object using the attribute interface, 
either o.__annotations__ or getattr(o, '__annotations__').  Accessing 
'__annotations__' through other mechanisms (e.g. looking in the class dict) is 
unsupported.
* Assume that o.__annotations__ is always either a dict or None.
* It's best to not assign to o.__annotations__.  But if you must, always set it 
to either a dict or None.
* Never delete o.__annotations__.
* Never modify o.__annotations__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

By the way, here's a tidbit I never got around to posting in c.l.p-d.

I noted in the conversation in January that attrs is an outlier here: it 
*doesn't* look in the class dict for __annotations__.  Instead, it has some 
complicated code where it asks the class for its annotations, then iterates 
over the __mro__ and asks every base class for *its* annotations.  If the class 
and a base class have the same class dict (using the "is" operator iirc) then 
attrs says "oh, that class doesn't have its own annotations, it's inheriting 
them" and reacts appropriately.

I emailed Hynek to ask him why he did it that way.  It turns out, he did that 
because at the time he didn't know you could peek in the class dict for 
__annotations__.  He literally had a todo item saying "change to looking in the 
class dict".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

> - Is it possible to create __annotations__ lazily? (IIRC in January we came 
> to a conclusion about this, something like yes for modules but for classes, 
> or the other way around?)

Maybe for modules, definitely not for classes.  The problem is that best 
practice for classes is to look in the class dict for __annotations__.  That 
sidesteps the inheritance problem.

Functions already lazily create a dict __annotations__ if not set.  And it's 
not stored in the function dict.  So nobody ever looks in the function dict for 
__annotations__.  That all works fine.

Would you prefer that *just classes* lazily create __annotations__?  It's not 
hard code to write, but I was being cautious.  "always set __annotations__ to 
an empty dict" is the easiest code to write and get correct.  And since modules 
are not all that numerous even in the largest projects, it seemed like this 
un-optimized approach would have a minimal contribution to the heat death of 
the universe.

It's also remotely possible that someone out there does look in the module dict 
for __annotations__, though I admit I've never seen it.  It seems like it'd be 
tempting to write your code that way though:

if isinstance(o, (type, types.ModuleType)):
ann = o.__dict__.get("__annotations__", None)
elif callable(o):
ann = o.__annotations__
else:
raise ValueError(f"{o!r} doesn't support annotations")


> - Why would __annotations__ ever be None?

Simply because it's the cheapest sensible way to indicate "this object has no 
annotations".  It would be nice if the world accepted __annotations__ being 
None to mean "no annotations".  But I don't think we're there.

I note that the function object has a special setter for __annotations__ 
(func_set_annotations()), since at least Python 3.1, which explicitly only 
allows setting __annotations__ to either a dict or None.  (I didn't have 3.0 
handy.)


> Why do you allow setting it to None?

Me?  I've never contributed code to Python that restricts the permissible types 
of values one is allowed to set on __annotations__.

Function objects are opinionated as mentioned (either dict or None), classes 
and modules have no opinion whatsoever, and allow you to set __annotations__ to 
any value.  That was all true long before I started working on annotations.


> Do we know of people ever write '__annotations__ = None' in their class or 
> write 'cls.__annotations__ = None'?

AFAIK I've never seen anyone set __annotations__ to None.


> - And where's your PR?

Not done yet.  I only started it last night, and there were a lot of test 
failures--it turns out, a *lot* of regression tests do a sanity check that 
__annotations__ isn't set on classes (and maybe modules too).  For example, I 
was surprised that test_opcodes has two such test failures.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Guido van Rossum]

Guido van Rossum  added the comment:

Sounds like a plan, I agree that the original design misfired here (we were 
probably worried about million-line code bases with tens of thousands of 
classes having to pay the price of tens of thousands of empty dicts, but I now 
think that was an unnecessary worry -- that should be a few megabytes on a very 
much larger total).

But given that you're not done yet:

- Is it possible to create __annotations__ lazily? (IIRC in January we came to 
a conclusion about this, something like yes for modules but for classes, or the 
other way around?)

- Why would __annotations__ ever be None? Why do you allow setting it to None? 
Do we know of people ever write '__annotations__ = None' in their class or 
write 'cls.__annotations__ = None'?

- And where's your PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

Preliminary patch is 17 lines.  Ah well.  I must be terrible at this!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Larry Hastings  added the comment:

Huh.  The sample code in my thread got goofed up somewhere along the way--maybe 
it's my fault, maybe it was done to me by some automated process.  Anyway, the 
example demonstrating classes inheriting annotations was meant to be formatted 
like this:

class A:
ax:int=3
class B(A):
pass
print(getattr(B, '__annotations__', {}))

Maybe some over-smart doodad decided that no paragraph would ever start with 
spaces, so they helpfully removed them?  Hmm.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43901] Add an empty annotations dict to all unannotated classes and modules

[Larry Hastings]

Change by Larry Hastings :


--
title: Add an empty annotations dict to all classes and modules -> Add an empty 
annotations dict to all unannotated classes and modules

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com