Here's another suggestion: PEP 593 introduced the `Annotated` type annotation. This could be used to annotate a TypeGuard like this:
`def is_str_list(val: List[object]) -> Annotated[bool, TypeGuard(List[str])]` Note that I used ( ) instead of [ ] for the TypeGuard, as it is no longer a type. This should fulfill all four requirements, but is a lot more verbose and therefore also longer. It would also be extensible for other annotations. For the most extensible approach both `-> TypeGuard(...)` and `-> Annotated[bool, TypeGuard(...)]` could be allowed, which would open the path for future non-type-annotations, which could be used regardless of whether the code is type-annotated. -- Adrian On February 14, 2021 2:20:14 PM GMT+01:00, Steven D'Aprano <st...@pearwood.info> wrote: >On Sat, Feb 13, 2021 at 07:48:10PM -0000, Eric Traut wrote: > >> I think it's a reasonable criticism that it's not obvious that a >> function annotated with a return type of `TypeGuard[x]` should return >> a bool. >[...] >> As Guido said, it's something that a developer can easily >> look up if they are confused about what it means. > >Yes, developers can use Bing and Google :-) > >But it's not the fact that people have to look it up. It's the fact that >they need to know that this return annotation is not what it seems, but >a special magic value that needs to be looked up. > >That's my objection: we're overloading the return annotation to be >something other than the return annotation, but only for this one >special value. (So far.) If you don't already know that it is special, >you won't know that you need to look it up to learn that its special. > > >> I'm open to alternative formulations that meet the following requirements: >> >> 1. It must be possible to express the type guard within the function >> signature. In other words, the implementation should not need to be >> present. This is important for compatibility with type stubs and to >> guarantee consistent behaviors between type checkers. > >When you say "implementation", do you mean the body of the function? > >Why is this a hard requirement? Stub files can contain function >bodies, usually `...` by convention, but alternatives are often useful, >such as docstrings, `raise NotImplementedError()` etc. > >https://mypy.readthedocs.io/en/stable/stubs.html > >I don't think that the need to support stub files implies that the type >guard must be in the function signature. Have I missed something? > > >> 2. It must be possible to annotate the input parameter types _and_ the >> resulting (narrowed) type. It's not sufficient to annotate just one or >> the other. > >Naturally :-) > >That's the whole point of a type guard, I agree that this is a truly >hard requirement. > > >> 3. It must be possible for a type checker to determine when narrowing >> can be applied and when it cannot. This implies the need for a bool >> response. > >Do you mean a bool return type? Sorry Eric, sometimes the terminology >you use is not familiar to me and I have to guess what you mean. > > >> 4. It should not require changes to the grammar because that would >> prevent this from being adopted in most code bases for many years. > >Fair enough. > > >> Mark, none of your suggestions meet these requirements. > >Mark's suggestion to use a variable annotation in the body meets >requirements 2, 3, and 4. As I state above, I don't think that >requirement 1 needs to be a genuinely hard requirement: stub files can >include function bodies. > >To be technically precise, stub functions **must** include function >bodies. It's just that by convention we use typically use `...` as the >body. > > >> Gregory, one of your suggestions meets these requirements: >> >> ```python >> def is_str_list(val: Constrains[List[object]:List[str]) -> bool: >> ... >> ``` > >That still misleadingly tells the reader (or naive code analysis >software) that parameter val is of type > > Contrains[List[object]:List[str]] > >whatever that object is, rather than what it *actually* is, namely >`List[object]`. I dislike code that misleads the reader. > > >> As for choosing the name of the annotation >[...] >> `TypeGuard` is the term that is used in other languages to describe >> this notion, so it seems reasonable to me to adopt this term > >Okay, this reasoning makes sense to me. Whether spelled as a decorator >or an annotation, using TypeGuard is reasonable. > > >> Steven, you said you'd like to explore a decorator-based formulation. >> Let's explore that. Here's what that it look like if we were to meet >> all of the above requirements. >> >> ```python >> @type_guard(List[str]) >> def is_str_list(val: List[object]) -> bool: ... >> ``` > >Okay. > > >I note that this could be easily extended to support narrowing in the >negative case as well: > > @type_guard(List[str], List[float]) > def is_str_list(val: List[Union[str, float]]) -> bool: ... > > >> The problem here, as I mention in the "rejected ideas" section of the >> PEP, is that even with postponed type evaluations (as described in PEP >> 563), the interpreter cannot postpone the evaluation of an expression >> if it's used as the argument to a decorator. That's because it's not >> being used as a type annotation in this context. > >Agreed. > > >> So while Mark is >> correct to point out that there has been a mechanism available for >> forward references since PEP 484, > >That was actually me who pointed out the quoting mechanism for forward >references. (Unless Mark also did so.) > > >> we've been trying to eliminate the >> use of quoted type expressions in favor of postponed evaluation. This >> would add a new case that can't be handled through postponed >> evaluation. Perhaps you still don't see that as a strong enough >> justification for rejecting the decorator-based formulation. I'm not >> entirely opposed to using a decorator here, but I think on balance >> that the `TypeGuard[x]` formulation is better. Once again, that's a >> subjective opinion. > >I understand the desire to minimize the use of quoted forward >references. But the cost to save two quote characters seems high: >changing an obvious and straight-forward return annotation to an >actively misleadingly special case. (Also, see below for the `Callable` >case.) > >I'm not convinced that forward references will be common. Did I miss >something, or are there no examples in the PEP that require a >forward-reference? > > > # Spam is not defined yet, so a forward reference is needed. > > def is_list_of_spam(values:List[object]) -> TypeGuard[List[Spam]]: > return all(isinstance(x, Spam) for x in values) > > # versus decorator > > @type_guard('List[Spam]') > def is_list_of_spam(values:List[object]) -> bool: > return all(isinstance(x, Spam) for x in values) > > >Of course, neither of these examples will actually work, because Spam >doesn't exist so you can't refer to it in the body. I don't get the >sense that this will require forward-references very often. At least not >often enough to justify obfuscating the return type. > >This obfuscation appears to have a critical consequence too. Please >correct me if I am wrong, but quoting your PEP: > >""" >In all other respects, TypeGuard is a distinct type from bool. It is >not a subtype of bool. Therefore, Callable[..., TypeGuard[int]] is not >assignable to Callable[..., bool]. >""" > >If I am reading this correctly that implies that if I define these >functions: > > >``` >def is_str_list(val: List[object]) -> TypeGuard[List[str]]: > return all(isinstance(x, str) for x in val) > >def my_filter(func:Callable[object, bool], values:List[object]) -> >List[object]: > return [x for x in values if func(x)] >``` > >the type checker would be expected to flag this as invalid: > > my_filter(is_str_list, ['a', 'b']) > > >If I have not misunderstood, surely this is a critical flaw with the >PEP? > >Eric, for what it's worth, I think that this will be an excellent >feature for type checkers, thank you for writing the PEP. It's just the >syntax gives me the willies: > >- special case handling of TypeGuard in a way that obfuscates the actual > return type; > >- actively misleads the reader, and any naive code analysis tools that > don't know about type guards; > >- prevents user type guards from being seen as `Callable[..., bool]` > even though they actually are. > >And the justification for eliminating decorators seems to be weak to me. > >However, I will give you one possible point in its favour: runtime >introspection of the annotations by typeguard-aware tools. If an >introspection tool is aware of the special meaning of `TypeGuard` in the >return annotation, then it is easy to introspect that value at runtime: > > # untested... > T = func.__annotations__['return'] > if T.startswith("TypeGuard"): > print("return type is bool") > print("narrows to type ...") # TODO: extract from T > else: > print(f"return type is {T}") > > >With a decorator, we would need some other solution for runtime >introspection of the type guard. That's easy enough to solve, the >obvious solution is to just add the information to the function as an >attribute. But it is one additional complication. > >Still, I think this is a minor issue, especially compared to the >critical issue of `Callable[..., bool]`. > >I think that the runtime introspection issue will probably rule out >Mark's "variable annotation in the body" idea. So considering all the >factors as I see them, including familiarity to TypeScript devs: > >* Decorator: +1 > >* Mark's variable annotation, if runtime introspection is neglected: +0.1 > >* Gregory's idea to annotate the parameter itself: -0.1 > >* The PEP's return value annotation: -0.2 > >* Mark's variable annotation, if runtime introspection is required: -1 > > >-- >Steve >_______________________________________________ >Python-Dev mailing list -- python-dev@python.org >To unsubscribe send an email to python-dev-le...@python.org >https://mail.python.org/mailman3/lists/python-dev.python.org/ >Message archived at >https://mail.python.org/archives/list/python-dev@python.org/message/OE6N443ASDFAH2KSBZ5BJX6UFNZKEAWM/ >Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KFKTZ6L5MI6S7KZY4W6PGZZWRR2PQTQF/ Code of Conduct: http://python.org/psf/codeofconduct/
