[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Thanks for your feedback, everybody!  It's now checked in.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:


New changeset 74613a46fc79cacc88d3eae4105b12691cd4ba20 by larryhastings in 
branch 'master':
bpo-43817: Add inspect.get_annotations(). (#25522)
https://github.com/python/cpython/commit/74613a46fc79cacc88d3eae4105b12691cd4ba20


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Ie debated about this with myself (and with a friend!) over the last few days, 
and I've concluded that evaluating strings by default is too opinionated for 
the standard library.  I've changed both inspect.signature() and 
inspect.get_annotations() so eval_str is False by default (and removed the 
ONLY_IF_STRINGIZED logic entirely).  Both functions will still raise an 
exception if eval_str=True, but this will only happen if the user explicitly 
requests evaluating the strings.

I've already updated the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Inada Naoki]

Inada Naoki  added the comment:

> Just to be clear: I would *not* want this new mode to be the *default* 
> behavior.  So far I think ONLY_IF_STRINGIZED is the best compromise for 
> default behavior.

I don't think ONLY_IF_STRINGIZED is the best compromise. I don't think it solve 
any issue.

```
if False:
from typing import List

def f1() -> 'List[str]':
pass

def f2(a:int) -> 'List[str]':
pass
```

In this example, both of help(f1) and help(f2) must show signature, not raise 
an error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Guido van Rossum]

Guido van Rossum  added the comment:

No, I want the *default* not to raise when eval() fails.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Guido van Rossum]

Guido van Rossum  added the comment:

I want to be as short as possible.

The user may have annotations because they are using mypy. They may be
using a library that calls inspect.signature() on some of their functions
to retrieve the parameter names conveniently.

All this has worked for years, the library never caring about annotations,
the user probably adding a few more annotations as they get more value out
of mypy.

Now one day, using Python 3.9, the user adds "from __future__ import
annotations" to their module, and removes the quotes from annotations
containing forward references or things imported under "if TYPE_CHECKING".

This works in 3.9. But you want to break it in 3.10. This is what I object
to.

I realize this also broke in the 3.10 alphas where PEP 563 was the default.
But that was rolled back for a reason! And we changed inspect.signature()
to call get_type_hints() (i.e. call eval() and break if it fails) because
it wasn't just affecting users PEP 563, it was affecting everyone who *was*
expecting a type object rather than a string in Parameter.annotation. That
bug also exists in 3.9, but only for users of PEP 563 (most of whom didn't
care about runtime inspection of annotations, so it didn't matter to them
that Parameter.annotation got a string.

You can draw the matrix of use cases and Python versions yourself.

Given that we're not going whole-hog on PEP 563 in 3.10, I think we should
either leave sleeping dogs lie, and continue to return a string as
Parameter.annotation if PEP 563 was in effect, or else we should at least
make inspect.signature() not fail if the string fails to evaluate. Note
that the version of inspect.signature() that I remember being checked into
3.10 did catch exceptions from get_type_hints() and used the original
__annotations__ dict (i.e. with all strings) if there was an eval error (or
any other error, IIRC). Please go look for it if you don't believe me (it
would have been rolled back last week in the Great Rollback).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

One final thought on that idea.  Part of the reason why I didn't go back to the 
drawing board and re-think the API was because I thought folks were pushing 
back on the idea of *default* behavior possibly raising exceptions.

If I misread it, and the pushback was "we want real Python values when we can 
get them, and strings when we can't, and it's acceptable if we have to 
explicitly request that behavior from inspect.signature() and 
inspect.get_annotations()", then hot diggity!, maybe we've finally found a 
compromise we can all agree on.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Just to be clear: I would *not* want this new mode to be the *default* 
behavior.  So far I think ONLY_IF_STRINGIZED is the best compromise for default 
behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

I like Eric's suggestion best of all.  I'd be willing to add a "silence errors 
on a case-by-case basis" flag to inspect.signature().  I imagine that would add 
a new field to the Parameter object (as Guido suggested) indicating which 
objects failed.  Would that be acceptable all around?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

> There may be a (deliberate? :-) misunderstanding. When I wrote about
> "you" inspecting code by a "3rd party" I meant that as a symmetric
> relationship -- the "you" could be a library and from the library's
> POV the "3rd party" could be you (or me).

I wasn't deliberately misunderstanding you.  And I understand what you're 
saying.  But the relationship isn't perfectly symmetric from a pragmatic 
perspective.

Let's say I write some code, and I also call into a third-party library.  I've 
annotated one of my objects, and the third-party library calls 
inspect.signature() on my object.  If my annotations are strings, and they 
aren't eval()uatable, then boom! it raises an exception, showing me a bug in my 
code, and I can fix it straight away.

On the other hand: let's say I write some code, and I call into a third-party 
library.  If the third-party library has an annotated object, and I call 
inspect.signature() on that object, and the annotations happen to be strings 
that aren't evaluatable, boom! it's showing me a bug in the third-party 
library.  Now I have a much larger headache: I have to notify this third-party 
vendor, convince them to fix the bug, wait for a new release, etc etc etc.  
(And in the meantime maybe I add eval_str=False to my inspect.signature() call.)

It's my assumption that the former scenario is far more likely than the latter, 
which is good news because it's way easier to deal with.  The latter scenario 
is plausible, and much more inconvenient, but I think it's far less likely.  If 
I'm wrong, and there are lots of third-party libraries with stringized 
annotations that could have errors lurking in them, I'd very much like to know 
that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Eric V. Smith]

Eric V. Smith  added the comment:

I'd like to see the default behavior be to raise an exception if eval fails on 
any annotation.

I think it's reasonable to provide a way to find out which specific keys have 
problems, but I don't think that should be the default. Wouldn't it be good 
enough to have a flag which says "just return me a dict which only has keys for 
items which don't contain errors"? Let's call it silence_errors for discussion 
sake. You could then figure out which ones contain errors by:

getattr(obj, "__annotations__", {}).keys() - get_annotations(obj, 
silence_errors=True).keys()

That is, silence_errors works on a per-key basis, not the call to 
get_annotations() as a whole.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

> I use inspect.signature for getting information about callables
> (third-party and first-party) in my type checker:
> https://github.com/quora/pyanalyze/blob/master/pyanalyze/arg_spec.py#L436. 
>  In that context, I'd much rather get string annotations that I can
> process myself later than get an exception if the annotations aren't
> valid at runtime.

This use case is what the eval_str parameter is for.  Since you're dealing 
specifically with type hints (as opposed to annotations generally), you don't 
really care whether you get strings or valid objects--you need to handle both.  
So in Python 3.10+ you can--and should!--call inspect.get_annotations() and 
inspect.signature() with eval_str=False.  If you do that you won't have any 
trouble.


Since I keep getting new proposals on how to suppress eval() errors in 
inspect.signature(), I think I need to zoom out and talk about why I don't want 
to do it at all.  Forgive me, but this is long--I'm gonna start from first 
principles.

Changing inspect.signature() so it calls eval() on annotations is a change in 
3.10.  And, due to the fact that it's possible (how likely? nobody seems to 
know) that there are malformed string annotations lurking in user code, this 
has the possibility of being a breaking change.

In order to handle this correctly, I think you need to start with a more 
fundamental question: are *stringized* annotations supposed to be a hidden 
implementation detail, and Python should present annotations as real values 
whether or not they were internally stringized?  Or: if the user adds "from 
__future__ import annotation" to their module, is this them saying they 
explicitly want their annotations as strings and they should always see them as 
strings?

(Again, this is specifically when the *language* turns your annotations into 
strings with the "from __future" import.  I think if the user enters 
annotations as strings, the language should preserve those annotations as 
strings, and that includes the library.)

It seems to me that the first answer is right.  PEP 563 talks a lot about 
"here's how you turn your stringized annotations back into objects".  In 
particular, it recommends calling typing.get_type_hints(), which AFAIK has 
always called eval() to turn string annotations back into objects.  It's worth 
noting here that, in both 3.9 and in the current Python trunk, 
typing.get_type_hints() doesn't catch exceptions.

Also, during the development of Python 3.10, during a time when stringized 
annotations had become the default behavior, inspect.signature() was changed to 
call typing.get_type_hints().  Presumably to have typing.get_type_hints() 
handle the tricky work of calling eval().

(I had problems with this specific approach.  "annotations" and "type hints" 
aren't the same thing, so having inspect.signature() e.g. wrap some annotations 
with Optional, and change None to NoneType, was always a mistake.  Also, 
typing.get_type_hints() was changed at this time to catch "Exception" and 
suppress *all* errors raised during the eval() call.  Happily both these 
changes have since been backed out.)

>From that perspective, *not* having inspect.signature() turn stringized 
>annotations back into strings from the very beginning was a bug.  And changing 
>inspect.signature() in Python 3.10 so it calls eval() on string annotations is 
>a bug fix.  So far folks seem to agree--the pushback I'm getting is regarding 
>details of my approach, not on the idea of doing it at all.


Now we hit our second question.  It's possible that inspect.signature() can't 
eval() every stringized annotation back into a Python value, due to a malformed 
annotation expression that wasn't caught at compile-time.  This means 
inspect.signature() calls that worked in 3.9 could potentially start failing in 
3.10.  How should we address it?

>From the perspective of "string annotations are a hidden implementation 
>detail, and users want to see real objects", I think the fact that it wasn't 
>already failing was also a bug.  If your annotations are malformed, surely you 
>want to know about it.  If you have a malformed annotation, and you don't turn 
>on stringized annotations, you get an exception when the annotation expression 
>is evaluated at module import time, in any Python version up to an including 
>3.10.  Stringized annotations delays this evaluation to when the annotations 
>are examined--which means the exception for a malformed expression gets 
>delayed too.  Which in turn means, from my perspective, the fact that 
>inspect.signature() didn't raise on malformed annotation expressions was a bug.

I just don't agree that silently changing the failed eval()'d string annotation 
into something else is the right approach.  There have been a bunch of 
proposals along these lines, and while I appreciate the creative contributions 
and the attempts at problem-solving, I haven't liked any of them.  But it's not 
that I 

[issue43817] Add inspect.get_annotations()

[Guido van Rossum]

Guido van Rossum  added the comment:

There may be a (deliberate? :-) misunderstanding. When I wrote about "you" 
inspecting code by a "3rd party" I meant that as a symmetric relationship -- 
the "you" could be a library and from the library's POV the "3rd party" could 
be you (or me).

Either way, given that inspect.signature() collects many disparate aspects of 
the parameters of a function, it seems a bad design for it to raise an 
exception if any one of those aspects of any one of those parameters doesn't 
match an expectation (other than an expectation enforced by the parser+compiler 
themselves, like the constraint that positional-only parameters must precede 
other types of parameters).

I looked at the code, and there's a special value that Parameter().annotation 
is set if there's no annotation (Parameter.empty). Maybe you can follow that 
model and add another special value for errors during evaluation?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

I agree with Guido that it's better to design inspect.signature to not throw an 
error for annotations that don't eval() cleanly.

I use inspect.signature for getting information about callables (third-party 
and first-party) in my type checker: 
https://github.com/quora/pyanalyze/blob/master/pyanalyze/arg_spec.py#L436. 
 In that context, I'd much rather get string annotations that I can process 
myself later than get an exception if the annotations aren't valid at runtime. 
In the former case I can still get useful information out of the signature even 
if I don't know how to process some annotations. For example, I'll still know 
what parameters exist even if I don't know what their types are. In the latter 
case, I don't get any useful signature data.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

> I'm not a big user of the inspect module, but I always thought that
> its use was so that you could look at a function (or other object)
> *produced by a 3rd party* and learn something about it.

That's interesting!  I always thought its main use was the opposite--it was so 
third parties could introspect *your* functions.  Like, Pydantic examines your 
function's signature so it can wrap it with runtime validation.  (Though in 
this specific case, I believe Pydantic examines __annotations__ directly.)  I 
also dimly recall a Python library that built COM bindings around your code, 
but I can't find it right now.

I can't remember ever writing application code that used inspect.signature() to 
inspect library functions--but I've certainly written library functions that 
used inspect.signature() to inspect application code.


> Asking for the signature is one such operation, and I'd be rather
> upset if I couldn't inspect the annotation if that 3rd party happened
> to have added a bad string annotation.

Do you think that's likely?  I'd be quite surprised if it was common for third 
party libraries to have string annotations--either manual or using automatic 
stringizing--in the first place.  I thought string annotations were really only 
used as part of type hints, and the type-hinted code was all internal code 
bases from large organizations, not third-party libraries.  But I concede I 
know little about it.

Following on to that, the annotations would have to be bad, which means again 
either someone made a typo or it was done deliberately.  The latter reason 
would be simply obnoxious if done by a third-party library--third-party 
libraries should not have the circular imports problem used to justifiy such 
practices.

I suppose third-party libraries are just as error-prone as anybody else, so if 
they were manually stringizing their annotations, it could happen there.  Which 
I agree would be annoying, just like any bug in a third-party library.  But I 
wouldn't agree this specific error is so special that we need to suppress it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Guido van Rossum]

Guido van Rossum  added the comment:

I'm not a big user of the inspect module, but I always thought that its use was 
so that you could look at a function (or other object) *produced by a 3rd 
party* and learn something about it.

Asking for the signature is one such operation, and I'd be rather upset if I 
couldn't inspect the annotation if that 3rd party happened to have added a bad 
string annotation. especially if I wasn't interested in the annotation part of 
the signature in the first place. (There's a lot of other useful info to be 
gleaned from it.)

OTOH I think it's fine for get_annotations() to raise by default if there's a 
bad annotation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

I keep thinking about it, and I think letting inspect.get_annotations() and 
inspect.signature() raise exceptions is the right API choice.

I note that that's what typing.get_type_hints() did in Python 3.9.  During the 
development of Python 3.10, this was changed; typing.get_type_hints() started 
catching Exception and returning the original annotation dictionary (with 
un-eval'd strings).  This change has been reverted, and as of the current 
moment typing.get_type_hints() no longer catches exceptions on eval().

I think there are two types of people who will have string annotations that 
throw an exception when eval'd:

1) People who made a typo or other mistake.
2) People who deliberately use undefined identifiers in their annotations, due 
to circular import / circular dependencies in their code bases.

The people who are saying "just catch the exception and let it pass silently" 
seem to be in group 2.  I suggest that people in group 2 are sophisticated 
enough to start passing in eval_str=False to inspect.signature().  And I think 
people in group 1 would want to be alerted to their mistake, rather than have 
the library silently catch the error and mysteriously change its behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Perhaps eval_str=ONLY_IF_STRINGIZED should also add the semantics "if 
evaluating any string fails, behave as if eval_str=false".  I would *not* 
propose adding that for eval_str=true.  But people keep asking for this.  Hmm.

The heuristic is a tricky thing.  That's why in my "PEP 1212" idea I proposed a 
way that a function like get_annotations() could determine unambiguously 
whether or not the annotations for an object were stringized.  (And, if we did 
do something like that in a future Python version, this would also change the 
"if evaluating any string fails" behavior I just proposed.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Inada Naoki]

Inada Naoki  added the comment:

>
> The difference between eval_str=True and eval_str=ONLY_IF_STRINGIZED:
>
> def foo(a:int, b:"howdy howdy"): ...
>
> inspect.get_annotations(foo, eval_str=True) throws an exception.
> inspect.get_annotations(foo, eval_str=ONLY_IF_STRINGIZED) returns {'a': int, 
> b: 'howdy howdy'}
>
> Type hints have a convention that string annotations are a "forward 
> declaration" and should be eval()uated.  Annotations don't have such a 
> convention--a string is a legal annotation, and is not required to be valid 
> Python.
>

For such use case, ONLY_IF_STRINGIZED thorows an exception for `def
foo(a: "howdy howdy")` anyway.
In such cases, they should use `eval_str=False`, or `eval_str=True`
*and* `return_str_when_eval_failed=True` option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

The difference between eval_str=True and eval_str=ONLY_IF_STRINGIZED:

def foo(a:int, b:"howdy howdy"): ...

inspect.get_annotations(foo, eval_str=True) throws an exception.
inspect.get_annotations(foo, eval_str=ONLY_IF_STRINGIZED) returns {'a': int, b: 
'howdy howdy'}

Type hints have a convention that string annotations are a "forward 
declaration" and should be eval()uated.  Annotations don't have such a 
convention--a string is a legal annotation, and is not required to be valid 
Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Inada Naoki]

Inada Naoki  added the comment:

>  I think Pydantic would prefer it, because Pydantic wants to see the real 
> objects at runtime, rather than the stringized annotations.

If so, why don't they use `eval_str=True`?

I can not find any use cases where `eval_str= ONLY_IF_ALL_STR` is better than 
`eval_str=True`.

On the other hand, I think "return string annotation instead of raising error 
when eval failed" option is useful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

For what it's worth: I changed the name to ONLY_IF_STRINGIZED in the PR.

Since I propose that it be the default, everyone who called 
inspect.get_annotations(), and inspect.signature(), would use it.  I think 
Pydantic would prefer it, because Pydantic wants to see the real objects at 
runtime, rather than the stringized annotations.

If stringized annotations were deprecated in the future, then eval_str should 
be deprecated at the same time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Inada Naoki]

Inada Naoki  added the comment:

I'm not sure `ONLY_IF_ALL_STR` is worth enough.

Will anyone use it after Python 3.10? inspect.signature()? Pydantic?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Time runs short for Python 3.10b1.  Can I get a review from anybody here, say 
over the weekend?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Eric V. Smith]

Eric V. Smith  added the comment:

> p.s. assuming you meant PEP 563, not PEP 573.

Yes. Someday I'll figure out this keyboard.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Change by Larry Hastings :


--
assignee:  -> larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

PR is up, passes all checks.  I think it's ready for the first round of reviews!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

I think it gets a little murkier when we talk about *annotations* vs *type 
hints*.  Type hints have a defined meaning for a string: a string is a sort of 
forward declaration, and you eval() the string to get the real value.  (Or, 
not, if you're comfortable working with the stringized version of the type 
hint.)  So typing.get_type_hints() calls eval() on *every* annotation value of 
type str.

But inspect.get_annotations() can't be so opinionated.  If the user entered a 
string as their annotation, it should assume they want the string to show up in 
the annotations dict.  This is why I'm trying to be so smart with the 
"eval_str" default value heuristic.

The text in the docs about a "future version of Python" is pursuant to my vague 
"PEP 1212" idea, which would let get_annotations() determine or not whether the 
annotations were stringized by the compiler.  Or whatever we wind up deciding 
to do for Python 3.11--which, as you say, will hopefully disambiguate this 
question.

p.s. assuming you meant PEP 563, not PEP 573.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Eric V. Smith]

Eric V. Smith  added the comment:

> To be honest, I've never been really sure if the idea of PEP 563 stringized 
> annotations are supposed to be a hidden implementation detail, or a 
> first-class concept that the user (and the standard library) is expected to 
> deal with.

This is an excellent question. My assumption with dataclasses has been that 
it's something the user is expected to deal with. It's a user-visible change, 
that all users of __annotations__ are going to have to deal with. I understood 
PEP 573 as saying: if you don't want to see a stringized annotation, call 
typing.get_type_hints() yourself. My assumption is that neither @dataclass nor 
anybody else is going to do that for you.

However, not everyone agreed. See for example issue 39442.

I do think that wherever we end up with __annotations__ in 3.11, we should be 
explicit about answering your question.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Change by Larry Hastings :


--
keywords: +patch
pull_requests: +24242
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/25522

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

When I proposed this new function, stringized annotations were the default 
behavior in Python 3.10, and there were two calls to typing.get_type_hints() in 
the standard library:

   * inspect.signature()
   * functools.singledispatch()

Now that stringized annotations are no longer the default in 3.10, we've 
reverted that change to inspect.signature().  The call is still present in 
functools.singledispatch(), but that's been there for a while--it's in 3.9.4 
for example.

It's worth considering changing inspect.signature() to use get_annotations().  
To be honest, I've never been really sure if the idea of PEP 563 stringized 
annotations are supposed to be a hidden implementation detail, or a first-class 
concept that the user (and the standard library) is expected to deal with.  My 
current get_annotations() implementation assumes the former, and eval()s the 
annotations back into real Python values if it thinks that's the right thing to 
do.  Personally I'd prefer that inspect.signature() un-stringized annotations 
for me--I would see that as a useful feature--but I could be completely wrong 
on this.

For now, I'm going to plan on modifying inspect.signature() so it calls 
get_annotations() as part of this PR.

(I'll also modify inspect.signature() to take a "eval_str" keyword-only 
parameter, and pass that through to get_annotations().  That gives the caller 
control over this un-stringizing feature, in case they need to turn it off.)


Łukasz: what about functools.singledispatch()?  IIUC, it uses 
typing.get_type_hints() to get the type of the first parameter of a function, 
so it can use that as an index into a dict mapping first-argument types to 
functions..  Since it's not a user-visible detail perhaps it doesn't matter.  
But it seems a little strange to me that it's using 
typing.get_type_hints()--ISTM it should be calling inspect.signature(), or 
possibly this proposed new inspect.get_annotations().  What do you think?

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Eric V. Smith]

Eric V. Smith  added the comment:

> And since there are definitely circumstances in which it can't return 
> __annotations__ directly, that indicates that it should never return 
> __annotations__ directly.

And the follow on here is that get_annotations() shouldn't set __annotations__ 
if it's None. I realize I'm just stating the obvious.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Just to round the bases: get_annotations() won't return an unmodified 
__annotations__ dict, but it *could* return a *consistent* dict.  It could keep 
a cache (lru or otherwise) of all responses so far.  I don't think that's what 
we want, I just thought I should point out the option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

> Are you saying the user would expect to be able to change __annotations__ my 
> modifying the dict they get back?

As the docs are currently written, it's ambiguous.

> Is it ever the case that the user can modify __annotations__ through the dict 
> that's returned? That is: does __annotations__ itself ever get returned?

Yes.  I could enumerate the cases in which that is true but I don't think it 
would shed light.  Suffice to say, as currently written, get_annotations() 
currently returns the original dict unmodified when it can, and returns a 
freshly-created dict when it can't.

> I think you'd either want __annotations__ returned all the time, or never 
> returned. Otherwise some cases could modify __annotations__, and some 
> couldn't.

I think you're right!  And since there are definitely circumstances in which it 
can't return __annotations__ directly, that indicates that it should never 
return __annotations__ directly.  Good call!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Eric V. Smith]

Eric V. Smith  added the comment:

> If o.__annotations__ is None, should this function set the empty dict on the 
> object?  That seems slightly too opinionated to me.  On the other hand, the 
> user would probably expect that they could change the dict they got back.

Are you saying the user would expect to be able to change __annotations__ my 
modifying the dict they get back?

Is it ever the case that the user can modify __annotations__ through the dict 
that's returned? That is: does __annotations__ itself ever get returned?

I think you'd either want __annotations__ returned all the time, or never 
returned. Otherwise some cases could modify __annotations__, and some couldn't.

If __annotations__ is never returned, then I wouldn't set __annotations__ in 
this case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Hmm.  If o.__annotations__ is None, should this function set the empty dict on 
the object?  That seems slightly too opinionated to me.  On the other hand, the 
user would probably expect that they could change the dict they got back.  (If 
Python shipped with a builtin "frozen dict", I suppose I could safely return 
one of those.)


Note that the danger of this happening should drop to about zero, assuming I 
push through my other small patch:

https://bugs.python.org/issue43901

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Larry Hastings  added the comment:

Just over twelve hours ago, the Python Steering Committee announced that 
stringized annotations would no longer be default behavior in Python 3.10.  
They will go back to being gated with "from __future__ import annotations".  I 
think we still need this function, but naturally the implementation will now be 
a bit different.

It's going to take time to change the default semantics of annotations back to 
the way they were in Python 3.9.  In the meantime, I've coded up a first draft 
of inspect.get_annotations(), written assuming stringized annotations are 
optional.  It's attached below.  I'll also paste in the definition and the 
docstring into the text box here for your reading convenience.

I assert that get_annotations() should definitely try to un-stringize 
stringized annotations by default.  The default behavior of eval_str is 
sliightly magical, but I'm not sure I can do anything any smarter (apart from 
"resist the temptation to guess").  Unsophisticated users will want to see real 
values, and it may be important to give sophisticated users control over 
whether or not eval() is called.

By the same token: if eval_str stays as part of the interface, I propose 
exposing it from the library functions that will call get_annotations():
  * inspect.signature()
  * functools.singledispatch()
That way, a project using stringized annotations due to gnarly circular 
imports/definitions problems could still use inspect.signature(), without 
having to worry about worrying about whether or not all the symbols were 
defined at runtime.

I'll interpret a lack of feedback as a sort of mumbled encouraging consensus.

-

def get_annotations(obj, globals=None, locals=None, *, 
eval_str=ONLY_IF_ALL_STR):
  
Compute the annotations dict for an object.

obj may be a callable, class, or module.
Passing in any other type of object raises TypeError.

This function handles several details for you:

  * Values of type str may be un-stringized using eval(),
depending on the value of eval_str.  This is intended
for use with stringized annotations
(from __future__ import annotations).
  * If obj doesn't have an annotations dict, returns an
empty dict.  (Functions and methods always have an
annotations dict; classes, modules, and other types of
callables may not.)
  * Ignores inherited annotations on classes.  If a class
doesn't have its own annotations dict, returns an empty dict.
  * Always, always, always returns a dict.

eval_str controls whether or not values of type str are replaced
with the result of calling eval() on those values:

  * If eval_str is true, eval() is called on values of type str.
  * If eval_str is false, values of type str are unchanged.
  * If eval_str is the special value inspect.ONLY_IF_ALL_STR,
which is the default, eval() is called on values of type str
only if *every* value in the dict is of type str.  This is a
heuristic; the goal is to only eval() stringized annotations.
(If, in a future version of Python, get_annotations() is able
to determine authoritatively whether or not an annotations
dict contains stringized annotations, inspect.ONLY_IF_ALL_STR
will use that authoritative source instead of the heuristic.)

globals and locals are passed in to eval(); see the documentation
for eval() for more information.  If globals is None,
get_annotations() uses a context-specific default value,
contingent on type(obj):

  * If obj is a module, globals defaults to obj.__dict__.
  * If obj is a class, globals defaults to
sys.modules[obj.__module__].__dict__.
  * If obj is a callable, globals defaults to obj.__globals__,
although if obj is a wrapped function (using
functools.update_wrapper()) it is first unwrapped.

--
Added file: https://bugs.python.org/file49968/get_annotations.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43817] Add inspect.get_annotations()

[Larry Hastings]

Change by Larry Hastings :


--
title: Add typing.get_annotations() -> Add inspect.get_annotations()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com