On Sat, Feb 13, 2021 at 07:48:10PM -0000, Eric Traut wrote: > I think it's a reasonable criticism that it's not obvious that a > function annotated with a return type of `TypeGuard[x]` should return > a bool. [...] > As Guido said, it's something that a developer can easily > look up if they are confused about what it means.
Yes, developers can use Bing and Google :-) But it's not the fact that people have to look it up. It's the fact that they need to know that this return annotation is not what it seems, but a special magic value that needs to be looked up. That's my objection: we're overloading the return annotation to be something other than the return annotation, but only for this one special value. (So far.) If you don't already know that it is special, you won't know that you need to look it up to learn that its special. > I'm open to alternative formulations that meet the following requirements: > > 1. It must be possible to express the type guard within the function > signature. In other words, the implementation should not need to be > present. This is important for compatibility with type stubs and to > guarantee consistent behaviors between type checkers. When you say "implementation", do you mean the body of the function? Why is this a hard requirement? Stub files can contain function bodies, usually `...` by convention, but alternatives are often useful, such as docstrings, `raise NotImplementedError()` etc. https://mypy.readthedocs.io/en/stable/stubs.html I don't think that the need to support stub files implies that the type guard must be in the function signature. Have I missed something? > 2. It must be possible to annotate the input parameter types _and_ the > resulting (narrowed) type. It's not sufficient to annotate just one or > the other. Naturally :-) That's the whole point of a type guard, I agree that this is a truly hard requirement. > 3. It must be possible for a type checker to determine when narrowing > can be applied and when it cannot. This implies the need for a bool > response. Do you mean a bool return type? Sorry Eric, sometimes the terminology you use is not familiar to me and I have to guess what you mean. > 4. It should not require changes to the grammar because that would > prevent this from being adopted in most code bases for many years. Fair enough. > Mark, none of your suggestions meet these requirements. Mark's suggestion to use a variable annotation in the body meets requirements 2, 3, and 4. As I state above, I don't think that requirement 1 needs to be a genuinely hard requirement: stub files can include function bodies. To be technically precise, stub functions **must** include function bodies. It's just that by convention we use typically use `...` as the body. > Gregory, one of your suggestions meets these requirements: > > ```python > def is_str_list(val: Constrains[List[object]:List[str]) -> bool: > ... > ``` That still misleadingly tells the reader (or naive code analysis software) that parameter val is of type Contrains[List[object]:List[str]] whatever that object is, rather than what it *actually* is, namely `List[object]`. I dislike code that misleads the reader. > As for choosing the name of the annotation [...] > `TypeGuard` is the term that is used in other languages to describe > this notion, so it seems reasonable to me to adopt this term Okay, this reasoning makes sense to me. Whether spelled as a decorator or an annotation, using TypeGuard is reasonable. > Steven, you said you'd like to explore a decorator-based formulation. > Let's explore that. Here's what that it look like if we were to meet > all of the above requirements. > > ```python > @type_guard(List[str]) > def is_str_list(val: List[object]) -> bool: ... > ``` Okay. I note that this could be easily extended to support narrowing in the negative case as well: @type_guard(List[str], List[float]) def is_str_list(val: List[Union[str, float]]) -> bool: ... > The problem here, as I mention in the "rejected ideas" section of the > PEP, is that even with postponed type evaluations (as described in PEP > 563), the interpreter cannot postpone the evaluation of an expression > if it's used as the argument to a decorator. That's because it's not > being used as a type annotation in this context. Agreed. > So while Mark is > correct to point out that there has been a mechanism available for > forward references since PEP 484, That was actually me who pointed out the quoting mechanism for forward references. (Unless Mark also did so.) > we've been trying to eliminate the > use of quoted type expressions in favor of postponed evaluation. This > would add a new case that can't be handled through postponed > evaluation. Perhaps you still don't see that as a strong enough > justification for rejecting the decorator-based formulation. I'm not > entirely opposed to using a decorator here, but I think on balance > that the `TypeGuard[x]` formulation is better. Once again, that's a > subjective opinion. I understand the desire to minimize the use of quoted forward references. But the cost to save two quote characters seems high: changing an obvious and straight-forward return annotation to an actively misleadingly special case. (Also, see below for the `Callable` case.) I'm not convinced that forward references will be common. Did I miss something, or are there no examples in the PEP that require a forward-reference? # Spam is not defined yet, so a forward reference is needed. def is_list_of_spam(values:List[object]) -> TypeGuard[List[Spam]]: return all(isinstance(x, Spam) for x in values) # versus decorator @type_guard('List[Spam]') def is_list_of_spam(values:List[object]) -> bool: return all(isinstance(x, Spam) for x in values) Of course, neither of these examples will actually work, because Spam doesn't exist so you can't refer to it in the body. I don't get the sense that this will require forward-references very often. At least not often enough to justify obfuscating the return type. This obfuscation appears to have a critical consequence too. Please correct me if I am wrong, but quoting your PEP: """ In all other respects, TypeGuard is a distinct type from bool. It is not a subtype of bool. Therefore, Callable[..., TypeGuard[int]] is not assignable to Callable[..., bool]. """ If I am reading this correctly that implies that if I define these functions: ``` def is_str_list(val: List[object]) -> TypeGuard[List[str]]: return all(isinstance(x, str) for x in val) def my_filter(func:Callable[object, bool], values:List[object]) -> List[object]: return [x for x in values if func(x)] ``` the type checker would be expected to flag this as invalid: my_filter(is_str_list, ['a', 'b']) If I have not misunderstood, surely this is a critical flaw with the PEP? Eric, for what it's worth, I think that this will be an excellent feature for type checkers, thank you for writing the PEP. It's just the syntax gives me the willies: - special case handling of TypeGuard in a way that obfuscates the actual return type; - actively misleads the reader, and any naive code analysis tools that don't know about type guards; - prevents user type guards from being seen as `Callable[..., bool]` even though they actually are. And the justification for eliminating decorators seems to be weak to me. However, I will give you one possible point in its favour: runtime introspection of the annotations by typeguard-aware tools. If an introspection tool is aware of the special meaning of `TypeGuard` in the return annotation, then it is easy to introspect that value at runtime: # untested... T = func.__annotations__['return'] if T.startswith("TypeGuard"): print("return type is bool") print("narrows to type ...") # TODO: extract from T else: print(f"return type is {T}") With a decorator, we would need some other solution for runtime introspection of the type guard. That's easy enough to solve, the obvious solution is to just add the information to the function as an attribute. But it is one additional complication. Still, I think this is a minor issue, especially compared to the critical issue of `Callable[..., bool]`. I think that the runtime introspection issue will probably rule out Mark's "variable annotation in the body" idea. So considering all the factors as I see them, including familiarity to TypeScript devs: * Decorator: +1 * Mark's variable annotation, if runtime introspection is neglected: +0.1 * Gregory's idea to annotate the parameter itself: -0.1 * The PEP's return value annotation: -0.2 * Mark's variable annotation, if runtime introspection is required: -1 -- Steve _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OE6N443ASDFAH2KSBZ5BJX6UFNZKEAWM/ Code of Conduct: http://python.org/psf/codeofconduct/
