I'm a +1 on using Annotated in this manner. Guido mentioned that it was intended for only third-parties though. I'd like to know more about why this isn't a good pattern for use by Python libraries.
On Sun, 2021-02-14 at 16:29 +0100, Adrian Freund wrote: > Here's another suggestion: > > PEP 593 introduced the `Annotated` type annotation. This could be > used to annotate a TypeGuard like this: > > `def is_str_list(val: List[object]) -> Annotated[bool, > TypeGuard(List[str])]` > > Note that I used ( ) instead of [ ] for the TypeGuard, as it is no > longer a type. > > This should fulfill all four requirements, but is a lot more verbose > and therefore also longer. > It would also be extensible for other annotations. > > For the most extensible approach both `-> TypeGuard(...)` and `-> > Annotated[bool, TypeGuard(...)]` could be allowed, which would open > the path for future non-type-annotations, which could be used > regardless of whether the code is type-annotated. > > > -- > Adrian > > On February 14, 2021 2:20:14 PM GMT+01:00, Steven D'Aprano > <st...@pearwood.info> wrote: > > On Sat, Feb 13, 2021 at 07:48:10PM -0000, Eric Traut wrote: > > > > > I think it's a reasonable criticism that it's not obvious that a > > > function annotated with a return type of `TypeGuard[x]` should > return > > > a bool. > > > > > [...] > > > As Guido said, it's something that a developer can easily > > > look up if they are confused about what it means. > > > > > > > Yes, developers can use Bing and Google :-) > > > > But it's not the fact that people have to look it up. It's the fact > that > > they need to know that this return annotation is not what it seems, > but > > a special magic value that needs to be looked up. > > > > That's my objection: we're overloading the return annotation to be > > something other than the return annotation, but only for this one > > special value. (So far.) If you don't already know that it is > special, > > you won't know that you need to look it up to learn that its > special. > > > > > > > I'm open to alternative formulations that meet the following > requirements: > > > > > > 1. It must be possible to express the type guard within the > function > > > signature. In other words, the implementation should not need to > be > > > present. This is important for compatibility with type stubs and > to > > > guarantee consistent behaviors between type checkers. > > > > > > > When you say "implementation", do you mean the body of the > function? > > > > Why is this a hard requirement? Stub files can contain function > > bodies, usually `...` by convention, but alternatives are often > useful, > > such as docstrings, `raise NotImplementedError()` etc. > > > > https://mypy.readthedocs.io/en/stable/stubs.html > > > > I don't think that the need to support stub files implies that the > type > > guard must be in the function signature. Have I missed something? > > > > > > > 2. It must be possible to annotate the input parameter types > _and_ the > > > resulting (narrowed) type. It's not sufficient to annotate just > one or > > > the other. > > > > > > > Naturally :-) > > > > That's the whole point of a type guard, I agree that this is a > truly > > hard requirement. > > > > > > > 3. It must be possible for a type checker to determine when > narrowing > > > can be applied and when it cannot. This implies the need for a > bool > > > response. > > > > > > > Do you mean a bool return type? Sorry Eric, sometimes the > terminology > > you use is not familiar to me and I have to guess what you mean. > > > > > > > 4. It should not require changes to the grammar because that > would > > > prevent this from being adopted in most code bases for many > years. > > > > > > > Fair enough. > > > > > > > Mark, none of your suggestions meet these requirements. > > > > > > > Mark's suggestion to use a variable annotation in the body meets > > requirements 2, 3, and 4. As I state above, I don't think that > > requirement 1 needs to be a genuinely hard requirement: stub files > can > > include function bodies. > > > > To be technically precise, stub functions **must** include function > > bodies. It's just that by convention we use typically use `...` as > the > > body. > > > > > > > Gregory, one of your suggestions meets these requirements: > > > > > > ```python > > > def is_str_list(val: Constrains[List[object]:List[str]) -> bool: > > > ... > > > ``` > > > > > > > That still misleadingly tells the reader (or naive code analysis > > software) that parameter val is of type > > > > Contrains[List[object]:List[str]] > > > > whatever that object is, rather than what it *actually* is, namely > > `List[object]`. I dislike code that misleads the reader. > > > > > > > As for choosing the name of the annotation > > > > > [...] > > > `TypeGuard` is the term that is used in other languages to > describe > > > this notion, so it seems reasonable to me to adopt this term > > > > > > > Okay, this reasoning makes sense to me. Whether spelled as a > decorator > > or an annotation, using TypeGuard is reasonable. > > > > > > > Steven, you said you'd like to explore a decorator-based > formulation. > > > Let's explore that. Here's what that it look like if we were to > meet > > > all of the above requirements. > > > > > > ```python > > > @type_guard(List[str]) > > > def is_str_list(val: List[object]) -> bool: ... > > > ``` > > > > > > > Okay. > > > > > > I note that this could be easily extended to support narrowing in > the > > negative case as well: > > > > @type_guard(List[str], List[float]) > > def is_str_list(val: List[Union[str, float]]) -> bool: ... > > > > > > > The problem here, as I mention in the "rejected ideas" section of > the > > > PEP, is that even with postponed type evaluations (as described > in PEP > > > 563), the interpreter cannot postpone the evaluation of an > expression > > > if it's used as the argument to a decorator. That's because it's > not > > > being used as a type annotation in this context. > > > > > > > Agreed. > > > > > > > So while Mark is > > > correct to point out that there has been a mechanism available > for > > > forward references since PEP 484, > > > > > > > That was actually me who pointed out the quoting mechanism for > forward > > references. (Unless Mark also did so.) > > > > > > > we've been trying to eliminate the > > > use of quoted type expressions in favor of postponed evaluation. > This > > > would add a new case that can't be handled through postponed > > > evaluation. Perhaps you still don't see that as a strong enough > > > justification for rejecting the decorator-based formulation. I'm > not > > > entirely opposed to using a decorator here, but I think on > balance > > > that the `TypeGuard[x]` formulation is better. Once again, that's > a > > > subjective opinion. > > > > > > > I understand the desire to minimize the use of quoted forward > > references. But the cost to save two quote characters seems high: > > changing an obvious and straight-forward return annotation to an > > actively misleadingly special case. (Also, see below for the > `Callable` > > case.) > > > > I'm not convinced that forward references will be common. Did I > miss > > something, or are there no examples in the PEP that require a > > forward-reference? > > > > > > # Spam is not defined yet, so a forward reference is needed. > > > > def is_list_of_spam(values:List[object]) -> TypeGuard[List[Spam]]: > > return all(isinstance(x, Spam) for x in values) > > > > # versus decorator > > > > @type_guard('List[Spam]') > > def is_list_of_spam(values:List[object]) -> bool: > > return all(isinstance(x, Spam) for x in values) > > > > > > Of course, neither of these examples will actually work, because > Spam > > doesn't exist so you can't refer to it in the body. I don't get the > > sense that this will require forward-references very often. At > least not > > often enough to justify obfuscating the return type. > > > > This obfuscation appears to have a critical consequence too. Please > > correct me if I am wrong, but quoting your PEP: > > > > """ > > In all other respects, TypeGuard is a distinct type from bool. It > is > > not a subtype of bool. Therefore, Callable[..., TypeGuard[int]] is > not > > assignable to Callable[..., bool]. > > """ > > > > If I am reading this correctly that implies that if I define these > > functions: > > > > > > ``` > > def is_str_list(val: List[object]) -> TypeGuard[List[str]]: > > return all(isinstance(x, str) for x in val) > > > > def my_filter(func:Callable[object, bool], values:List[object]) -> > > List[object]: > > return [x for x in values if func(x)] > > ``` > > > > the type checker would be expected to flag this as invalid: > > > > my_filter(is_str_list, ['a', 'b']) > > > > > > If I have not misunderstood, surely this is a critical flaw with > the > > PEP? > > > > Eric, for what it's worth, I think that this will be an excellent > > feature for type checkers, thank you for writing the PEP. It's just > the > > syntax gives me the willies: > > > > - special case handling of TypeGuard in a way that obfuscates the > actual > > return type; > > > > - actively misleads the reader, and any naive code analysis tools > that > > don't know about type guards; > > > > - prevents user type guards from being seen as `Callable[..., > bool]` > > even though they actually are. > > > > And the justification for eliminating decorators seems to be weak > to me. > > > > However, I will give you one possible point in its favour: runtime > > introspection of the annotations by typeguard-aware tools. If an > > introspection tool is aware of the special meaning of `TypeGuard` > in the > > return annotation, then it is easy to introspect that value at > runtime: > > > > # untested... > > T = func.__annotations__['return'] > > if T.startswith("TypeGuard"): > > print("return type is bool") > > print("narrows to type ...") # TODO: extract from T > > else: > > print(f"return type is {T}") > > > > > > With a decorator, we would need some other solution for runtime > > introspection of the type guard. That's easy enough to solve, the > > obvious solution is to just add the information to the function as > an > > attribute. But it is one additional complication. > > > > Still, I think this is a minor issue, especially compared to the > > critical issue of `Callable[..., bool]`. > > > > I think that the runtime introspection issue will probably rule out > > Mark's "variable annotation in the body" idea. So considering all > the > > factors as I see them, including familiarity to TypeScript devs: > > > > * Decorator: +1 > > > > * Mark's variable annotation, if runtime introspection is > neglected: +0.1 > > > > * Gregory's idea to annotate the parameter itself: -0.1 > > > > * The PEP's return value annotation: -0.2 > > > > * Mark's variable annotation, if runtime introspection is required: > -1 > > > > > > _______________________________________________ > > Python-Dev mailing list -- python-dev@python.org > > To unsubscribe send an email to python-dev-le...@python.org > > https://mail.python.org/mailman3/lists/python-dev.python.org/ > > Message archived at https://mail.python.org/archives/list/python-d > > e...@python.org/message/KFKTZ6L5MI6S7KZY4W6PGZZWRR2PQTQF/ > > Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UBNPEPAB2NBHY6KGDHF5SXMDVDPZH77U/ Code of Conduct: http://python.org/psf/codeofconduct/
