[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 3bc801960655ea265599805eac24173164b511a6 by Matt Bogosian in 
branch '3.10':
[3.10] bpo-46581: Propagate private vars via _GenericAlias.copy_with (GH-31061) 
(GH-31821)
https://github.com/python/cpython/commit/3bc801960655ea265599805eac24173164b511a6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Matt Bogosian]

Change by Matt Bogosian :


--
pull_requests: +29919
stage: backport needed -> patch review
pull_request: https://github.com/python/cpython/pull/31821

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +Jelle Zijlstra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Alex Waygood]

Change by Alex Waygood :


--
stage: patch review -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 32bf3597922ac3f613989582afa2bff43bea8a2f by Matt Bogosian in 
branch 'main':
bpo-46581: Propagate private vars via _GenericAlias.copy_with (GH-31061)
https://github.com/python/cpython/commit/32bf3597922ac3f613989582afa2bff43bea8a2f


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Matt Bogosian]

Change by Matt Bogosian :


--
keywords: +patch
nosy: +mbogosian
nosy_count: 4.0 -> 5.0
pull_requests: +29243
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31061

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Matt B]

Matt B  added the comment:

Thanks, @kj! Fantastic education and insight! I'm sad that I needed you as an 
interpreter but very grateful you were around to provide the interpretation. 
Working on a patch now….

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Ken Jin]

Ken Jin  added the comment:

> In short, I will do my best, but I suspect I will need patience and guidance 
> in arriving something acceptable.

Yeah no worries. typing.py is really complex for many historical reasons. I 
still don't grasp all of it and probably never will.

> what about _AnnotatedAlias, _SpecialGenericAlias, _UnionGenericAlias, or 
> _strip_annotations?

They shouldn't need modification. And there's an elegant reason why. Currently 
ParamSpec is only valid in Generic, Callable and Concatenate. We don't care 
about any other types. As long as ParamSpec appears in their __parameters__ and 
the copy_with of those 3 have the correct settings.

Well what about when they're nested? The other types blindly copy over whatever 
they see in __parameters__ of any nested types. So even if you don't pass those 
settings to their copy_with, it doesn't matter (well of course, until someone 
else uses this setting and expands its scope, then proceeds to trip over it :(. 
this seems like a potential trap affecting PEP 646's runtime.)

>>> X = Concatenate[int, ParamSpec('P')]
>>> Container[X]
typing.Container[typing.Concatenate[int, ~P]]
>>> Container[X].copy_with(X).__parameters__
(~P,)
^ Works even though this uses _SpecialGenericAlias, which doesn't properly pass 
in the settings!

> I can't find any tests that deal with copy_with directly, so I'm assuming its 
> use is stressed via higher level code paths, but it's not clear what use 
> cases that method is supposed to serve.

Yeah. If you search for "copy_with" in typing.py. You'll find that it's called 
in __getitem__. Now __getitem__ is used when we subscript an already 
subscripted type. What I mean is:

T = TypeVar('T')
X = List[T] (__class_getitem__ is called, returning a new genericalias object)
X[int] (__getitem__ is called, returning a new genericalias object)

We need copy_with because we want to create a full "copy" of all the args into 
the new GenericAlias object and not face weird problems with mutability. That 
specific use case is tested super often.

> _ConcatenateGenericAlias.copy_with, on the other hand, appears to return a 
> tuple rather than a type until it falls back to the parent implementation. 
> What does one do with that?

Frankly, I was quite lost too until I saw that it was part of bpo-44791, which 
deals with special special (undefined) cases in PEP 612 and Concatenate. 
Basically for a very weird Concatenate, the core dev there has chosen to return 
a tuple of types instead of another type. Let's just ignore those strange tuple 
paths, and only care about the path using the parent implementation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Matt B]

Matt B  added the comment:

I am happy to attempt a patch, but I don't understand what's going on with 
_ConcatenateGenericAlias. Or rather, I don't fully understand the various 
copy_with semantics. This code is *very* hard to follow.

Patching _GenericAlias.copy_with seems relatively straightforward:

 def copy_with(self, params):
-return self.__class__(self.__origin__, params, name=self._name, 
inst=self._inst)
+return self.__class__(self.__origin__, params, name=self._name, 
inst=self._inst,
+  _typevar_types=self._typevar_types,
+  _paramspec_tvars=self._paramspec_tvars)

_ConcatenateGenericAlias.copy_with, on the other hand, appears to return a 
tuple rather than a type until it falls back to the parent implementation. What 
does one do with that?

Also, what about _AnnotatedAlias, _SpecialGenericAlias, _UnionGenericAlias, or 
_strip_annotations? Do any of those need to be modified?

I can't find any tests that deal with copy_with directly, so I'm assuming its 
use is stressed via higher level code paths, but it's not clear what use cases 
that method is supposed to serve. The good news is that its use is confined to 
typing.py. The bad news is that file gives little insight to those who aren't 
already experts in that territory.

In short, I will do my best, but I suspect I will need patience and guidance in 
arriving something acceptable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Ken Jin]

Ken Jin  added the comment:

Wow! Thanks, that's an interesting find. My hunch is that we should be passing 
in _typevar_types and _paramspec_tvars in the copy_with of _GenericAlias and 
_ConcatenateGenericAlias. Inconsistent copy_with could trigger subtle bugs in 
ForwardRefs and get_type_hints. I can reproduce a semi-bug right now:

>>> P = ParamSpec('P')
>>> Callable[P, int].__parameters__
(~P,)
>>> Callable[P, int].copy_with((P,int))
typing.Callable[~P, int]
>>> Callable[P, int].copy_with((P,int)).__parameters__
()
^ This shouldn't be empty!!

Would you like to submit a patch for this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Nikita Sobolev]

Change by Nikita Sobolev :


--
nosy: +sobolevn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Alex Waygood]

Change by Alex Waygood :


--
keywords:  -patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Alex Waygood]

Change by Alex Waygood :


--
pull_requests:  -29201

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +gvanrossum, kj -miss-islington
stage: patch review -> 
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Matt B]

Matt B  added the comment:

Filed by request: 
https://github.com/python/cpython/pull/26091#issuecomment-1024900261

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[miss-islington]

Change by miss-islington :


--
keywords: +patch
nosy: +miss-islington
nosy_count: 1.0 -> 2.0
pull_requests: +29201
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26091

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46581] _typevar_types and _paramspec_tvars are missing from _GenericAlias.copy_with

[Matt B]

New submission from Matt B :

c55ff1b352f8b82184f80d9dea220e832691acfc was submitted to fix #44098 and added 
the _typevar_types and _paramspec_tvars properties to _GenericAlias. However, 
those properties continue to be omitted from _GenericAlias.copy_with[1].

Further, typing.py is fairly intricate, which makes it hard to understand if 
that is the only place those properties are absent. A more careful review/audit 
may be in order.

[1]: 
https://github.com/python/cpython/blob/8b1b27f1939cc4060531d198fdb09242f247ca7c/Lib/typing.py#L1069-L1070

--
components: Library (Lib)
messages: 412144
nosy: posita
priority: normal
severity: normal
status: open
title: _typevar_types and _paramspec_tvars are missing from 
_GenericAlias.copy_with
versions: Python 3.10, Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com