> > And what about PEP544 (protocols), which is being drafted? The PEP seems > to aim for having type objects that represent duck-typing > protocols/interfaces. Checking whether a protocol is implemented by an > object or type is clearly a useful thing to do at runtime, but it is not > really clear if isinstance would be a guaranteed feature for PEP544 > Protocols. > > So one question is, is it possible to draw the lines between what works > with isinstance and what doesn't, and between what details are checked by > isinstance and what aren't? -- Or should insinstance be reserved for a more > limited purpose, and add another check function, say `implements(...)`, > which would perhaps guarantee some answer for all combinations of object > and type? >
I'm guessing to implement PEP 544, many of the `__instancecheck__` and `__subclasscheck__` methods in `typing.py` would need to be updated to check the `__annotations__` of the class of the object it's passed against its own definition, (covered in this section <https://www.python.org/dev/peps/pep-0544/#runtime-decorator-and-narrowing-types-by-isinstance> of the PEP). I've been somewhat surprised that many of the `__instancecheck__` implementations do not work at runtime, even when the implementation would be trivial (e.g. for `Union`), or would not have subtle edge cases due to immutability (e.g. for `Tuple`, which cannot be used for checking parameterized instances). This seems like counterintuitive behavior that would be straightforward to fix, unless there are subtleties & edge cases I'm missing. If people are amenable to updating those cases, I'd be interested in submitting a patch to that effect. Best, Lucas On Sat, Jun 24, 2017 at 12:42 PM, Koos Zevenhoven <k7ho...@gmail.com> wrote: > There has been some discussion here and there concerning the differences > between runtime types and static types (mypy etc.). What I write below is > not really an idea or proposal---just a perspective, or a topic that people > may want to discuss. Since the discussion on this is currently very fuzzy > and scattered and not really happening either AFAICT (I've probably missed > many discussions, though). Anyway, I thought I'd give it a shot: > > Clearly, there needs to be some sort of distinction between runtime > classes/types and static types, because static types can be more precise > than Python's dynamic runtime semantics. For example, Iterable[int] is an > iterable that contains integers. For a static type checker, it is clear > what this means. But at runtime, it may be impossible to figure out whether > an iterable is really of this type without consuming the whole iterable and > checking whether each yielded element is an integer. Even that is not > possible if the iterable is infinite. Even Sequence[int] is problematic, > because checking the types of all elements of the sequence could take a > long time. > > Since things like isinstance(it, Iterable[int]) cannot guarantee a proper > answer, one easily arrives at the conclusion that static types and runtime > classes are just two separate things and that one cannot require that all > types support something like isinstance at runtime. > > On the other hand, there are many runtime things that can or could be done > using (type) annotations, for example: > > Multidispatch (example with hypothetical syntax below): > > @overload > def concatenate(parts: Iterable[str]) -> str: > return "".join(parts) > > @overload > def concatenate(parts: Iterable[bytes]) -> bytes: > return b"".join(parts) > > @overload > def concatenate(parts: Iterable[Iterable]) -> Iterable: > return itertools.chain(*parts) > > or runtime type checking: > > @check_types > def load_from_file(filename: Union[os.PathLike, str, bytes]): > with open(filename) as f: > return do_stuff_with(f.read()) > > which would automatically give a nice error message if, say, a file object > is given as argument instead of a path to a file. > > However useful (and efficient) these things might be, the runtime type > checks are problematic, as discussed above. > > Furthermore, other differences between runtime and static typing may > emerge (or have emerged), which will complicate the matter further. For > instance, the runtime __annotations__ of classes, modules and functions may > in some cases contain something completely different from what a type > checker thinks the type should be. > > These and other incompatibilities between runtime and static typing will > create two (or more) different kinds of type-annotated Python: > runtime-oriented Python and Python with static type checking. These may be > incompatible in both directions: a static type checker may complain about > code that is perfectly valid for the runtime folks, and code written for > static type checking may not be able to use new Python techniques that make > use of type hints at runtime. There may not even be a fully functional > subset of the two "languages". Different libraries will adhere to different > standards and will not be compatible with each other. The split will be > much worse and more difficult to understand than Python 2 vs 3, peoples > around the world will suffer like never before, and programming in Python > will become a very complicated mess. > > > One way of solving the problem would be that type annotations are only a > static concept, like with stubs or comment-based type annotations. This > would also be nice from a memory and performance perspective, as evaluating > and storing the annotations would not occupy memory (although both issues > and some more might be nicely solved by making the annotations lazily > ealuated). However, leaving out runtime effects of type annotations is not > the approach taken, and runtime introspection of annotations seems to have > some promising applications as well. And for many cases, the traditional > Python class actually acts very nicely as both the runtime and static type. > > So if type annotations will be both for runtime and for static checking, > how to make everything work for both static and runtime typing? > > Since a writer of a library does not know what the type hints will be used > for by the library users, it is very important that there is only one way > of making type annotations which will work regardless of what the > annotations are used for in the end. This will also make it much easier to > learn Python typing. > > Regarding runtime types and isinstance, let's look at the Iterable[int] > example. For this case, there are a few options: > > 1) Don't implement isinstance > > This is problematic for runtime uses of annotations. > > 2) isinstance([1, '2', 'three'], Iterable[int]) returns True > > This is in fact now the case. This is ok for many runtime situations, but > lacks precision compared to the static version. One may want to distinguish > between Iterable[int] and Iterable[str] at runtime (e.g. the multidispatch > example above). > > 3) Check as much as you can at runtime > > There could be something like Reiterable, which means the object is not > consumed by iterating over it, so one could actually check if all elements > are instances of int. This would be useful in some situations, but not > available for every object. Furthermore, the check could take an arbitrary > amount of time so it is not really suitable for things like multidispatch > or some matching constructs etc., where the performance overhead of the > type check is really important. > > 4) Do a deeper check than in (2) but trust the annotations > > For example, an instance of a class that has a method like > > def __iter__(self) -> Iterator[int]: > some code > > could be identified as Iterable[int] at runtime, even if it is not > guaranteed that all elements are really integers. > > On the other hand, an object returned by > > def get_ints() -> Iterable[int]: > some code > > does not know its own annotations, so the check is difficult to do at > runtime. And of course, there may not be annotations available. > > 5) Something else? > > > And what about PEP544 (protocols), which is being drafted? The PEP seems > to aim for having type objects that represent duck-typing > protocols/interfaces. Checking whether a protocol is implemented by an > object or type is clearly a useful thing to do at runtime, but it is not > really clear if isinstance would be a guaranteed feature for PEP544 > Protocols. > > So one question is, is it possible to draw the lines between what works > with isinstance and what doesn't, and between what details are checked by > isinstance and what aren't? -- Or should insinstance be reserved for a more > limited purpose, and add another check function, say `implements(...)`, > which would perhaps guarantee some answer for all combinations of object > and type? > > I'll stop here---this email is probably already much longer than a single > email should be ;) > > -- Koos > > > -- > + Koos Zevenhoven + http://twitter.com/k7hoven + > > _______________________________________________ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > >
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
