I'm 100% agree with Łukasz and Brett. +1 and thanks for writing this PEP. INADA Naoki <songofaca...@gmail.com>
On Fri, Nov 3, 2017 at 2:00 AM, Brett Cannon <br...@python.org> wrote: > > > On Thu, 2 Nov 2017 at 08:46 Steven D'Aprano <st...@pearwood.info> wrote: >> >> On Wed, Nov 01, 2017 at 03:48:00PM -0700, Lukasz Langa wrote: >> >> > PEP: 563 >> > Title: Postponed Evaluation of Annotations >> >> > This PEP proposes changing function annotations and variable annotations >> > so that they are no longer evaluated at function definition time. >> > Instead, they are preserved in ``__annotations__`` in string form. >> >> This means that now *all* annotations, not just forward references, are >> no longer validated at runtime and will allow arbitrary typos and >> errors: >> >> def spam(n:itn): # now valid >> ... >> >> Up to now, it has been only forward references that were vulnerable to >> that sort of thing. Of course running a type checker should pick those >> errors up, but the evaluation of annotations ensures that they are >> actually valid (not necessarily correct, but at least a valid name), >> even if you happen to not be running a type checker. That's useful. >> >> Are we happy to live with that change? > > > I would say "yes" for two reasons. One, if you're bothering to provide type > hints then you should be testing those type hints. So as you pointed out, > Steve, that will be caught at that point. > > Two, code editors with auto-completion will help prevent this kind of typo. > Now I would never suggest that we design Python with expectations of what > sort of tooling people have available, but in this instance it will help. It > also feeds into a question you ask below... > >> >> >> >> > Rationale and Goals >> > =================== >> > >> > PEP 3107 added support for arbitrary annotations on parts of a function >> > definition. Just like default values, annotations are evaluated at >> > function definition time. This creates a number of issues for the type >> > hinting use case: >> > >> > * forward references: when a type hint contains names that have not been >> > defined yet, that definition needs to be expressed as a string >> > literal; >> >> After all the discussion, I still don't see why this is an issue. >> Strings makes perfectly fine forward references. What is the problem >> that needs solving? Is this about people not wanting to type the leading >> and trailing ' around forward references? > > > I think it's mainly about the next point you ask about... > >> >> >> >> > * type hints are executed at module import time, which is not >> > computationally free. >> >> True; but is that really a performance bottleneck? If it is, that should >> be stated in the PEP, and state what typical performance improvement >> this change should give. >> >> After all, if we're going to break people's code in order to improve >> performance, we should at least be sure that it improves performance :-) > > > The cost of constructing some of the objects used as type hints can be very > expensive and make importing really expensive (this has been pointed out by > Lukasz previously as well as Inada-san). By making Python itself not have to > construct objects from e.g. the 'typing' module at runtime, you then don't > pay a runtime penalty for something you're almost never going to use at > runtime anyway. > >> >> >> >> > Postponing the evaluation of annotations solves both problems. >> >> Actually it doesn't. As your PEP says later: >> >> > This PEP is meant to solve the problem of forward references in type >> > annotations. There are still cases outside of annotations where >> > forward references will require usage of string literals. Those are >> > listed in a later section of this document. >> >> So the primary problem this PEP is designed to solve, isn't actually >> solved by this PEP. > > > I think the performance bit is really the big deal here. > > And as I mentioned earlier, if you turn all of your type hints into strings, > you lose auto-completion/intellisense which is a shame. > > I think there's also a benefit here of promoting the fact that type hints > are not a runtime thing, they are a static analysis thing. By requiring the > extra step to convert from a string to an actual object, it helps get the > point across that type hints are just bits of metadata for tooling and not > something you're expected really interact with at runtime unless you have a > really good reason to. > > So I'm +1 on the idea, but the __future__ statement is a bit too generic for > me. I would prefer something like `from __future__ import > annotation_strings` or `annotations_as_strings`. > > -Brett > >> >> >> (See Guido's comments, quoted later.) >> >> >> >> > Implementation >> > ============== >> > >> > In Python 4.0, function and variable annotations will no longer be >> > evaluated at definition time. Instead, a string form will be preserved >> > in the respective ``__annotations__`` dictionary. Static type checkers >> > will see no difference in behavior, >> >> Static checkers don't see __annotations__ at all, since that's not >> available at edit/compile time. Static checkers see only the source >> code. The checker (and the human reader!) will no longer have the useful >> clue that something is a forward reference: >> >> # before >> class C: >> def method(self, other:'C'): >> ... >> >> since the quotes around C will be redundant and almost certainly left >> out. And if they aren't left out, then what are we to make of the >> annotation? Will the quotes be stripped out, or left in? >> >> In other words, will method's __annotations__ contain 'C' or "'C'"? That >> will make a difference when the type hint is eval'ed. >> >> >> > If an annotation was already a string, this string is preserved >> > verbatim. >> >> That's ambiguous. See above. >> >> >> > Annotations can only use names present in the module scope as postponed >> > evaluation using local names is not reliable (with the sole exception of >> > class-level names resolved by ``typing.get_type_hints()``). >> >> Even if you call get_type_hints from inside the function defining the >> local names? >> >> def function(): >> A = something() >> def inner(x:A)->int: >> ... >> d = typing.get_type_hints(inner) >> return (d, inner) >> >> I would expect that should work. Will it? >> >> >> > For code which uses annotations for other purposes, a regular >> > ``eval(ann, globals, locals)`` call is enough to resolve the >> > annotation. >> >> Let's just hope nobody doing that has allowed any tainted strings to >> be stuffed into __annotations__. >> >> >> > * modules should use their own ``__dict__``. >> >> Which is better written as ``vars()`` with no argument, I believe. Or >> possibly ``globals()``. >> >> >> > If a function generates a class or a function with annotations that >> > have to use local variables, it can populate the given generated >> > object's ``__annotations__`` dictionary directly, without relying on >> > the compiler. >> >> I don't understand this paragraph. >> >> >> > The biggest controversy on the issue was Guido van Rossum's concern >> > that untokenizing annotation expressions back to their string form has >> > no precedent in the Python programming language and feels like a hacky >> > workaround. He said: >> > >> > One thing that comes to mind is that it's a very random change to >> > the language. It might be useful to have a more compact way to >> > indicate deferred execution of expressions (using less syntax than >> > ``lambda:``). But why would the use case of type annotations be so >> > all-important to change the language to do it there first (rather >> > than proposing a more general solution), given that there's already >> > a solution for this particular use case that requires very minimal >> > syntax? >> >> I agree with Guido's concern here. A more general solution would >> (hopefully!) be like a thunk, and might allow some interesting >> techniques unrelated to type checking. Just off the top of my head, say, >> late binding of default values (without the "if arg is None: arg = []" >> trick). >> >> >> > A few people voiced concerns that there are libraries using annotations >> > for non-typing purposes. However, none of the named libraries would be >> > invalidated by this PEP. They do require adapting to the new >> > requirement to call ``eval()`` on the annotation with the correct >> > ``globals`` and ``locals`` set. >> >> Since this is likely to be a common task for such libraries, can we have >> a evaluate_annotations() function to do this, rather than have everyone >> reinvent the wheel? >> >> def func(arg:int): >> ... >> >> evaluate_annotations(func) >> assert func.__annotations__['arg'] is int >> >> It could be a decorator, as well as modifying __annotations__ in place. >> >> I imagine something with a signature like this: >> >> def evaluate_annotations( >> obj:Union[Function, Class], >> globals:Dict=None, >> locals:Dict=None >> )->Union[Function, Class]: >> """Evaluate the __annotations__ of a function, or recursively >> a class and all its methods. Replace the __annotations__ in >> place. Returns the modified argument, making this suitable as >> a decorator. >> >> If globals is not given, it is taken from the function.__globals__ >> or class.__module__ if available. If locals is not given, it >> defaults to the current locals. >> """ >> >> >> -- >> Steve >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/brett%40python.org > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com > _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
