[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

More recent versions of typing have added some helper functions that could be 
useful here, like typing.get_origin and typing.get_args. I'm going to close 
this issue because I don't think there's anything actionable.

--
nosy: +Jelle Zijlstra
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Pekka Klärck]

Pekka Klärck  added the comment:

You are obviously right with how `__instancecheck__` and `__subclasscheck__` 
work. We'd either need something like `__rinstancecheck__` and 
`__rsubclasscheck__` or `isinstance` and `issubclass` needed to handle this 
using `types.resolve_bases`, `__origin__`, or something else.

It's unfortunate that `__origin__` cannot be considered to be part of the 
stable API and that no other suitable API exists. I don't want to add an 
external dependency only to handle this situation, so I guess I'll just use 
`__origin__` with Python 3.7+. Hopefully it isn't changed in 3.7 minor versions 
and hopefully a public API exists in 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

> but even then types in the typing could themselves implement 
> `__instancecheck__` and `__subclasscheck__` and retain the old behavior.

It doesn't work that way. `__instancecheck__` and `__subclasscheck__` tweaks 
the behaviour of superclass (i.e. the right argument) in `isinstance()` and 
`issubclass()`. This is how `isinstance([], typing.Iterable)` works, you can't 
use the same to tweak `isinstance(typing.Iterable, type)`.

> Hopefully someone with more insight on this can comment my worries. Perhaps 
> the attribute should also be documented as discussed earlier: 
> https://github.com/python/typing/issues/335

No, it is not safe to use it and will not be documented. You missed the point 
of my previous post, the idea is to add public wrappers in typing that will 
hide `__origin__` (or whatever else) as an implementation detail. Using 
`__origin__` is OK however as a *temporary* measure, if you don't want to use 
`typing_inspect` in the meantime.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Pekka Klärck]

Pekka Klärck  added the comment:

My concerns with the behavior of `__origin__` possibly changing in the future 
seem to valid. In Python 3.5 and 3.6 the behavior is 

List.__origin__ is None
List[int].__origin__ is List

while in Python 3.7

List.__origin is list
List[int].__origin__ is list

Is it likely that this is going to change again or can I rely on the current 
behavior with Python 3.7+?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Pekka Klärck]

Pekka Klärck  added the comment:

While studying the types in the typing module more, I noticed they have a 
special `__origin__` attribute which seems to contain the "real" type they 
represent. I was able to make my type conversion code to work by adding these 
lines:

if hasattr(type_, '__origin__'):
type_ = type_.__origin__

All our tests pass with this simple fix, but I'm slightly worried using it 
because `__origin__` doesn't seem to be documented. This means I'm not sure is 
my usage OK and, more importantly, makes me worried that another change in 
typing changes the behavior or removes the attribute altogether. Hopefully 
someone with more insight on this can comment my worries. Perhaps the attribute 
should also be documented as discussed earlier: 
https://github.com/python/typing/issues/335

I'd also be a little bit happier with the above fix if I could write it like

if isinstance(type_, typing.SomeBaseType):
type_ = type_.__origin__

but apparently types in the typing module don't have any public base class. I 
guess odds that some unrelated class would have `__origin__` defined is small 
enough that using `hasattr(type_, '__origin__')` is safe.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Pekka Klärck]

Pekka Klärck  added the comment:

Thanks for the PEP-560 reference. It explains the reasoning for the underlying 
changes, performance, and also mentions backwards incompatibility problems, 
including `issubclass(List[int], List)` causing a TypeError. It doesn't mention 
that `issubclass(List, list)` also raises a TypeError, though, nor that 
`isinstance(List, type)` now returns False.

I understand changing the implementation for performance reason, but I don't 
understand why that would require changing the behavior of `isinstance` and 
`issubclass`. The PEP explicitly mentions that the new `types.resolved_base` 
isn't called by them without explaining why. I guess that could be for 
performance reasons, but even then types in the typing could themselves 
implement `__instancecheck__` and `__subclasscheck__` and retain the old 
behavior. Or is there some actual reason for changing the behavior?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

It was a deliberate decision. You can find some motivation in PEP 560, and yes 
we used provisional status here. It was a hard decision, but we decided that 
giving up few percent of backwards compatibility is a reasonable price for up 
to 5x performance boost.

It looks like some of your problems may be solved by 
https://github.com/ilevkivskyi/typing_inspect (use `pip install 
typing_inspect`) that aims at providing cross-version runtime introspection of 
typing objects by carefully wrapping some "hidden" internal API.

There is a plan to include most used part of typing_inspect in typing itself, 
see for example https://github.com/python/typing/issues/570.
(it is hard to give an estimate about when, I really want to do this soon, but 
just don't have time).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
components: +Library (Lib)
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +gvanrossum, levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Pekka Klärck]

Pekka Klärck  added the comment:

Basically I'd like to get answers to these two questions:

1. Are the changes deliberate and covered by the fact that typing is a 
provisional module, or could the old functionality be restored?

2. If we cannot get the old functionality back, is there some other way to 
reliable detect is an annotation a type defined in the typing module? This 
should cover both the `List` and `List[int]` cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34568] Types in `typing` not anymore instances of `type` or subclasses of "real" types

[Pekka Klärck]

New submission from Pekka Klärck :

= Introduction =

In Python 3.5 and 3.6 types defined in the typing module are instances of 
`type` and also subclasses of the "real" type they represent. For example, both 
`isinstance(typing.List, type)` and `issubclass(typing.List, list)` return 
true. In Python 3.7 the former returns false and the latter causes a TypeError. 
I could find anything related to these changes in the Python 3.7 release notes 
or from the documentation of the typing module.

I explain my use case and the problems these changes have caused below.

= Use case =

I'm implementing automatic argument conversion to Robot Framework, a generic 
open source test automation framework, based on function annotations. The idea 
is that if a user has defined a keyword like

def example(arg: int):
# ...

we can convert argument passed in plain text test data like

Example42

into the correct type automatically. For more details see this issue in our 
tracker:
https://github.com/robotframework/robotframework/issues/2890


= Problem 1 =

I have implemented converters for different types and use annotations to find 
out the expected type for each argument. To exclude non-type annotations, my 
code uses `isinstance(annotation, type)` but in Python 3.7 this excludes also 
types defined in the typing module.

I could apparently use `isinstance(annoation, (type, typing._GenericAlias))`, 
but touching private parts like is fragile and feels wrong in general.

= Problem 2 =

Each converter I've implemented is mapped to a certain type (e.g. `list`) and, 
when applicable, also to an abc (e.g. `collections.abc.MutableSequence`). When 
finding a correct converter for a certain type, the code uses an equivalent of 
`issubclass(type_, (converter.type, converter.abc))`. In Python 3.5 and 3.6 
this works also if the used type is defined in the typing module but with 
Python 3.7 it causes a TypeError.

I guess I could handle the types in the typing module by explicitly mapping 
converters also to these types (e.g. `typing.List`) and then using something 
like `type_ is converter.typing`. The problem is that although it would work 
with types like `List`, it wouldn't work if types are used like `List[int]`.

--
messages: 324518
nosy: pekka.klarck
priority: normal
severity: normal
status: open
title: Types in `typing` not anymore instances of `type` or subclasses of 
"real" types

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com