[Python-Dev] Re: PEP 563: get_type_hints should use closure when available
I think this would just perpetuate misunderstanding about how things work.
You're proposing to make it so that changing the body of the function (in
particular, removing `y: T`) would affect the interpretation of the type
annotation. That's just wrong.
On Wed, Feb 3, 2021 at 12:47 PM Caleb Donovick
wrote:
> > Here we want to know what the 'C' refers to in g's annotations. But g's
> closures are no help (unless g's body were to also reference C, which is
> not likely in typical code).
>
> I understand the limitation. I was proposing changing
> https://github.com/python/cpython/blob/3.9/Lib/typing.py#L1431-L1433 to
> use the `nonlocal_vars` instead of `globalns` for the default localns (or
> possibly a chainmap of the two). I agree that it may only work on a small
> fraction of code but the performance cost seems pretty minimal especially
> for a function like `get_type_hints`, so why not? But I trust judgement,
> if you think that creating an apparent inconsistency around when
> `get_type_hints` will work and when it won't is not worth being able to
> resolve a few extra cases it probably isn't.
>
> - Caleb Donovick
>
> On Wed, Feb 3, 2021 at 8:55 AM Guido van Rossum wrote:
>
>> Sadly this won't work, because the annotations are "evaluated" (or not)
>> in the surrounding scope, not in the function's scope. Example:
>> ```
>> >>> def f(a: int):
>> ... class C: pass
>> ... def g(b: C): pass
>> ... return g
>> ...
>> >>> g = f(1)
>> >>> g.__annotations__
>> {'b': 'C'}
>> >>> g.__closure__
>> >>>
>> ```
>> Here we want to know what the 'C' refers to in g's annotations. But g's
>> closures are no help (unless g's body were to also reference C, which is
>> not likely in typical code).
>>
>> --Guido
>>
>> On Tue, Feb 2, 2021 at 6:10 PM Caleb Donovick
>> wrote:
>>
>>> The discussion around PEP 649 got me thinking about what I believe is
>>> the largest downside to PEP 563: the inability to evaluate annotations
>>> created with closures. While this is in general unavoidable, if the type
>>> is ever referenced in an annotated function (including as an annotation) it
>>> should be resolvable via `__closure__`.
>>>
>>> For example:
>>> ```
>>> from __future__ import annotations
>>> import typing
>>> def gen(T):
>>> def f(x: T):
>>> y: T = ...
>>> return f
>>>
>>> f = gen(int)
>>> nonlocal_vars = {
>>> var : cell.cell_contents
>>> for var, cell in zip(f.__code__.co_freevars, f.__closure__)
>>> }
>>> assert typing.get_type_hints(f, localns=nonlocal_vars) == {'x': int}
>>> ```
>>>
>>> I would just open a PR to have `get_type_hints` attempt to resolve
>>> closure variables by default. However, this would require an update to PEP
>>> 563 and I don't know what the protocol is there.
>>>
>>> - Caleb Donovick
>>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/L2XH2EIFCEPT5DQRPRFR6WYEM6M2LM2V/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> *Pronouns: he/him **(why is my pronoun here?)*
>>
[Python-Dev] Re: PEP 563: get_type_hints should use closure when available
> Here we want to know what the 'C' refers to in g's annotations. But g's
closures are no help (unless g's body were to also reference C, which is
not likely in typical code).
I understand the limitation. I was proposing changing
https://github.com/python/cpython/blob/3.9/Lib/typing.py#L1431-L1433 to use
the `nonlocal_vars` instead of `globalns` for the default localns (or
possibly a chainmap of the two). I agree that it may only work on a small
fraction of code but the performance cost seems pretty minimal especially
for a function like `get_type_hints`, so why not? But I trust judgement,
if you think that creating an apparent inconsistency around when
`get_type_hints` will work and when it won't is not worth being able to
resolve a few extra cases it probably isn't.
- Caleb Donovick
On Wed, Feb 3, 2021 at 8:55 AM Guido van Rossum wrote:
> Sadly this won't work, because the annotations are "evaluated" (or not) in
> the surrounding scope, not in the function's scope. Example:
> ```
> >>> def f(a: int):
> ... class C: pass
> ... def g(b: C): pass
> ... return g
> ...
> >>> g = f(1)
> >>> g.__annotations__
> {'b': 'C'}
> >>> g.__closure__
> >>>
> ```
> Here we want to know what the 'C' refers to in g's annotations. But g's
> closures are no help (unless g's body were to also reference C, which is
> not likely in typical code).
>
> --Guido
>
> On Tue, Feb 2, 2021 at 6:10 PM Caleb Donovick
> wrote:
>
>> The discussion around PEP 649 got me thinking about what I believe is the
>> largest downside to PEP 563: the inability to evaluate annotations created
>> with closures. While this is in general unavoidable, if the type is ever
>> referenced in an annotated function (including as an annotation) it should
>> be resolvable via `__closure__`.
>>
>> For example:
>> ```
>> from __future__ import annotations
>> import typing
>> def gen(T):
>> def f(x: T):
>> y: T = ...
>> return f
>>
>> f = gen(int)
>> nonlocal_vars = {
>> var : cell.cell_contents
>> for var, cell in zip(f.__code__.co_freevars, f.__closure__)
>> }
>> assert typing.get_type_hints(f, localns=nonlocal_vars) == {'x': int}
>> ```
>>
>> I would just open a PR to have `get_type_hints` attempt to resolve
>> closure variables by default. However, this would require an update to PEP
>> 563 and I don't know what the protocol is there.
>>
>> - Caleb Donovick
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/L2XH2EIFCEPT5DQRPRFR6WYEM6M2LM2V/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
>
[Python-Dev] Re: PEP 563: get_type_hints should use closure when available
Sadly this won't work, because the annotations are "evaluated" (or not) in
the surrounding scope, not in the function's scope. Example:
```
>>> def f(a: int):
... class C: pass
... def g(b: C): pass
... return g
...
>>> g = f(1)
>>> g.__annotations__
{'b': 'C'}
>>> g.__closure__
>>>
```
Here we want to know what the 'C' refers to in g's annotations. But g's
closures are no help (unless g's body were to also reference C, which is
not likely in typical code).
--Guido
On Tue, Feb 2, 2021 at 6:10 PM Caleb Donovick
wrote:
> The discussion around PEP 649 got me thinking about what I believe is the
> largest downside to PEP 563: the inability to evaluate annotations created
> with closures. While this is in general unavoidable, if the type is ever
> referenced in an annotated function (including as an annotation) it should
> be resolvable via `__closure__`.
>
> For example:
> ```
> from __future__ import annotations
> import typing
> def gen(T):
> def f(x: T):
> y: T = ...
> return f
>
> f = gen(int)
> nonlocal_vars = {
> var : cell.cell_contents
> for var, cell in zip(f.__code__.co_freevars, f.__closure__)
> }
> assert typing.get_type_hints(f, localns=nonlocal_vars) == {'x': int}
> ```
>
> I would just open a PR to have `get_type_hints` attempt to resolve closure
> variables by default. However, this would require an update to PEP 563 and
> I don't know what the protocol is there.
>
> - Caleb Donovick
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/L2XH2EIFCEPT5DQRPRFR6WYEM6M2LM2V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
[Python-Dev] Re: PEP 563: get_type_hints should use closure when available
On 2/2/2021 8:37 PM, Caleb Donovick wrote:
The discussion around PEP 649 got me thinking about what I believe is
the largest downside to PEP 563: the inability to evaluate annotations
created with closures. While this is in general unavoidable, if
the type is ever referenced in an annotated function (including as an
annotation) it should be resolvable via `__closure__`.
For example:
```
from __future__ import annotations
import typing
def gen(T):
def f(x: T):
y: T = ...
return f
f = gen(int)
nonlocal_vars = {
var : cell.cell_contents
for var, cell in zip(f.__code__.co_freevars, f.__closure__)
}
assert typing.get_type_hints(f, localns=nonlocal_vars) == {'x': int}
```
I would just open a PR to have `get_type_hints` attempt to resolve
closure variables by default. However, this would require an update to
PEP 563 and I don't know what the protocol is there.
Once a PEP is accepted and implemented, the PEP document itself is
frozen and not updated. The Python language and docs and the CPython
implementation are still subject to what is hopefully further progress.
You are free to open an thread on python-ideas list or an issue on bpo,
optionally with a PR, or both. It would be good to reference the PEP
and note whether this idea appears there or not. Further discussion of
the idea itself would depend on whether is was a rejected alternative, a
deferred addition, or just mentioned.
A speculative PR for an idea not yet accepted in principle may be
rejected but may help gain acceptance.
PS: I don't use type hints yet and have not much opinion about about
your idea (it naively seems plausible), so the above is a generic answer.
--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/FQLVHTIMDEI6IPO5FFL2ZIUKZC5RQK4U/
Code of Conduct: http://python.org/psf/codeofconduct/
