On Thu, Nov 2, 2017 at 3:45 PM, Steven D'Aprano <st...@pearwood.info> wrote:
> On Wed, Nov 01, 2017 at 03:48:00PM -0700, Lukasz Langa wrote:
>
> > This PEP proposes changing function annotations and variable annotations
> > so that they are no longer evaluated at function definition time.
> > Instead, they are preserved in ``__annotations__`` in string form.
>
> This means that now *all* annotations, not just forward references, are
> no longer validated at runtime and will allow arbitrary typos and
> errors:
>
> def spam(n:itn): # now valid
> ...
>
> Up to now, it has been only forward references that were vulnerable to
> that sort of thing. Of course running a type checker should pick those
> errors up, but the evaluation of annotations ensures that they are
> actually valid (not necessarily correct, but at least a valid name),
> even if you happen to not be running a type checker. That's useful.
>
> Are we happy to live with that change?
>
Within functions misspellings won't be caught until you invoke a function:
def spam(s):
return itn(s) # no error unless spam() is called
We've lived with this for a long time and generally people seem to be happy
with it. The vast majority of code in non-trivial programs (where type
annotations are useful) tends to be within functions, so this will only
slightly increase the number of things that won't be caught without running
tests (or running the type checker).
As type checking has become the main use case for annotations, using
annotations without a type checker is fast becoming a marginal use case.
Type checkers can easily and reliably validate that names in annotations aren't
misspelled.
> * forward references: when a type hint contains names that have not been
> > defined yet, that definition needs to be expressed as a string
> > literal;
>
> After all the discussion, I still don't see why this is an issue.
> Strings makes perfectly fine forward references. What is the problem
> that needs solving? Is this about people not wanting to type the leading
> and trailing ' around forward references?
>
Let's make a thought experiment. What if every forward reference would
require special quoting? Would Python programmers be happy with this? Say,
let's use ! as a suffix to mark a forward reference. They make perfectly
fine forward references. They are visually pretty unobtrusive (I'm not
suggesting $ or other ugly perlisms):
def main():
args = parse_args!() # A forward reference
do_stuff!(args) # Explicit is better than implicit
def parse_args():
...
def do_stuff(args):
...
Of course, I'm not seriously proposing this, but this highlights the fact
that in normal code forward references "just work" (at least usually), and
if we'd require a special quoting mechanism to use them anywhere, Python
would look uglier and more inconsistent. Nobody would be happy with this
change, even though you'd only have to type a single ! character extra --
that's not a lot work, right?
I think that the analogy is reasonable. In type checked code annotations
are one of most widely used language features -- it's quite possible to
have annotations for almost every function in a code base. This is not a
marginal feature, and people expect commonly used features to feel polished
and usable, not inconsistent and hacky. It's quite possible that the first
type annotated experiment a user writes requires the use of forward
references, and this gives a pretty bad first impression -- not unlike how
the ! forward reference would make the first impression of using
non-type-checked Python pretty awkward.
Here are more arguments why literal escapes are a usability problem:
1) It's hard to predict when string quotes are needed. Real-world large
code bases tend to have a lot of import cycles, and string literal escapes
are often needed within import cycles. However, they aren't always needed.
To annotate code correctly, you frequently need to understand how the file
you are editing is related to other modules in terms of import cycle
structure. In large code bases this can be very difficult to keep in your
head, so basically adding forward references becomes a matter of
tweak-until-it-works. So either each time you write an annotation, you can
look at how imports are structured -- to see whether a particular type
needs to be quoted -- or you can guess and hope for the best. This is a
terrible user experience and increases cognitive load significantly. Our
goal should not be to just have something that technically 'works', as this
is a very low standard. I want Python to be easy to use, intuitive and
elegant. I don't expect that anybody who has annotated large code bases
could consider string literal forward references to be any of those.
2) It's one of the top complaints from users. Even a non-user with a basic
understanding of mypy told me what amounts to "Python doesn't have real
static typing; forward references make it obvious that types are just an
afterthought".
3) It's not intuitive for many programmers, as string literals aren't used
frequently in Python for quoting other kinds of forward references. It may
be intuitive for programmers with a deep Python understanding, but they are
a minority. Other mainstream languages with type annotations don't use
quoting for forward references. C requires forward references to be
declared (but only once per type, not on every use), but I don't think that
C is a good model for Python anyway.
4) If you move chunks of code around, suddenly you may have to update your
forward references -- some quotes won't be needed any more, and new ones
may be required.
5) Some editors and other tools highlight string literals with a color
different from other type annotations, making them look ugly and out of
place. Some might fix this eventually, but you can argue that the current
behavior is reasonable, since forward references are just ordinary string
literals at runtime.
> This PEP is meant to solve the problem of forward references in type
> > annotations. There are still cases outside of annotations where
> > forward references will require usage of string literals. Those are
> > listed in a later section of this document.
>
> So the primary problem this PEP is designed to solve, isn't actually
> solved by this PEP.
>
Forward references in type annotations account for the vast majority of
forward references. Everything else is pretty marginal and typically
involves more advanced type system features. We can probably come up with
some data on this. Also, the other contexts are more clearly just regular
Python expressions, so the requirement to use forward references is
arguably less surprising. Consider this example:
FooList = List[Foo] # Foo only gets defined below
class Foo:
...
I think that even a pretty rudimentary understanding of how Python works
should be enough to understand that the above won't work at runtime without
quoting. However, this is less obvious:
class A:
...
def copy(self) -> 'A': # But wasn't A already declared above?
return A(...) # And here we can refer to A directly
Also, forward references already only work sometimes in non-type-checked
Python, so the proposed change would still be kind of consistent with how
things work now. Example:
def f():
return A() # Probably ok?
a = A() # Not ok
f() # Not ok
class A: ...
f() # Ok
Static checkers don't see __annotations__ at all, since that's not
> available at edit/compile time. Static checkers see only the source
> code. The checker (and the human reader!) will no longer have the useful
> clue that something is a forward reference:
>
A static checker really doesn't benefit from knowing that something is a
forward reference. Mypy, for example, doesn't need to know if something is
a forward reference to decide what it refers to. As I discussed above,
knowing whether something is a forward reference can actually be less than
helpful for a human reader as well.
Jukka
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
