Thanks for thinking more deeply about how we can make type annotations for callables more user friendly.
The intersection between the syntax for general Python code and type annotations has been a topic of discussion within the Steering Council (on and off) for quite some time. We intend to provide more clarity about our thinking, probably in the form of an informational PEP, around this topic, and my apologies because I haven’t had time to draft it yet. But since this came up, I want to at least say that we are unanimous in our belief that the type annotation language must also be standard Python syntax. In other words, we are not in favor of letting type annotation syntax deviate from Python syntax. As a effect of this, the PSC does not intend to issue a blanket delegation of type-focused PEPs. Of course, individual typing PEPs can be delegated just like any other PEP, but especially where syntax change proposals are made, I expect that the PSC will want keep final decision making. This means that any PEP which proposes syntactic changes to support typing features must also address the implications of those syntax changes for the general Python language. PEP 646 (Variadic Generics) is a good example of this. Early on we recognized that the proposed syntax change had implications for Python, and we asked the PEP authors to address those implications for Python, which they added: https://www.python.org/dev/peps/pep-0646/#grammar-changes This is the kind of analysis the PSC needs in order to weigh the cost and benefits of making such changes to Python. TL;RA - One Syntax to Rule Them All Cheers, -Barry > On Oct 7, 2021, at 09:41, S Pradeep Kumar <gohan...@gmail.com> wrote: > > Hello all, > > Typing-sig has been discussing user-friendly syntax for the type used to > represent callables. [1] Since this affects the Python language syntax, we > wanted to get some high-level feedback from you before putting up a detailed > PEP. > > TL;DR: We want to propose syntax for Callable types, e.g., `(int, str) -> > bool` instead of `typing.Callable[[int, str], bool]`. Questions: 1. Are there > concerns we need to keep in mind about such a syntax change? 2. Should we > propose syntax for additional features in the same PEP or in a future PEP? > > > # Motivation > > Guido has pointed out that `Callable` is one of the most frequently used type > forms but has the least readable syntax. For example, say we have a callback > `formatter` that accepts a record and a list of permissions: > > ```python > def print_purchases( > user, > formatter, # <-- callback > ): > <...> > output = formatter(record, permissions) > print(output) > ``` > > To give it a type, we currently have to write: > > ```python > from typing import Callable > > def print_purchases( > user: User, > formatter: Callable[[PurchaseRecord, List[AuthPermission]], > FormattedItem] # Hard to read. > ) -> None: > > <...> > output = formatter(record, permissions) > print(output) > ``` > > `Callable` can be hard to understand for new users since it doesn't resemble > existing function signatures and there can be multiple square brackets. It > also requires an import from `typing`, which is inconvenient. Around 40% of > the time [2], users just give up on precisely typing the parameters and > return type of their callbacks and just leave it as a blank Callable. In > other cases, they don't add a type for their callback at all. Both mean that > they lose the safety guarantees from typing and leave room for bugs. > > We believe that adding friendly syntax for Callable types will improve > readability and encourage users to type their callbacks more precisely. Other > modern, gradually-typed languages like TypeScript (JS), Hack (PHP), etc. have > special syntax for Callable types. > > (As a precedent, PEP 604 recently added clean, user-friendly syntax for the > widely-used `Union` type. Instead of importing `Union` and writing `expr: > Union[AddExpr, SubtractExpr], we can just write `expr: AddExpr | > SubtractExpr`.) > > > # Proposal and Questions > > We have a two-part proposal and questions for you: > > 1. Syntax to replace Callable > > After a lot of discussion, there is strong consensus in typing-sig about > adding syntax to replace Callable. So, the above example would be written as: > ```python > def print_purchases( > user: User, > formatter: (PurchaseRecord, List[AuthPermission]) -> FormattedItem, > ) -> None: > > <...> > output = formatter(record, permissions) > print(output) > ``` > This feels more natural and succinct. > > Async callbacks currently need to be written as > ``` > from typing import Callable > > async_callback: Callable[[HttpRequest], Awaitable[HttpResponse]] > ``` > > With the new syntax, they would be written as > ``` > async_callback: async (HttpRequest) -> HttpResponse > ``` > which again seems more natural. There is similar syntax for the type of > decorators that pass on *args and **kwargs to the decorated function. > > Note that we considered and rejected using a full def-signature syntax like > ```` > (record: PurchaseRecord, permissions: List[AuthPermission], /) -> > FormattedItem > ```` > because it would be more verbose for common cases and could lead to subtle > bugs; more details in [3]. > > The Callable type is also usable as an expression, like in type aliases > `IntOperator = (int, int) -> int` and `cast((int) -> int, f)` calls. > > **Question 1**: Are there concerns we should keep in mind about such a syntax > proposal? > > > > 2. Syntax for callback types beyond Callable > > `Callable` can't express the type of all possible callbacks. For example, it > doesn't support callbacks where some parameters have default values: > `formatter(record)` (the user didn't pass in `permissions`). It *is* possible > to express these advanced cases using Callback Protocols (PEP 544) [4] but it > gets verbose. > > There are two schools of thought on typing-sig on adding more syntax on top > of (1): > > (a) Some, including Guido, feel that it would be a shame to not have syntax > for core Python features like default values, keyword arguments, etc. > > One way to represent default values or optionally name parameters would > be: > > ``` > # permissions is optional > formatter: (PurchaseRecord, List[AuthPermission]=...) -> FormattedItem > > # permissions can be called using a keyword argument. > formatter: (PurchaseRecord, permissions: List[AuthPermission]) -> > FormattedItem > ``` > > There are also alternative syntax proposals. > > (b) Others want to wait till we have more real-world experience with the > syntax in (1). > > The above cases occur < 5% of the time in typed or untyped code [5]. And > the syntax in (1) is forward-compatible with the additional proposals. So we > could add them later if needed or leave them out, since we can always use > callback protocols. > > **Question 2**: Do you have preferences either way? Do we propose (1) alone > or (1) + (2)? > > > Once we get some high-level feedback here, we will draft a PEP going into the > details for various use cases. > > Best, > Pradeep Kumar Srinivasan > Steven Troxler > Eric Traut > > PS: We've linked to more details below. Happy to provide more details as > needed. > > [1]: typing-sig thread about the proposal: > https://mail.python.org/archives/list/typing-...@python.org/message/JZLYRAXJV34WAV5TKEOMA32V7ZLPOBFC/ > [2]: Stats about loosely-typed Callables: > https://github.com/pradeep90/annotation_collector#typed-projects---callable-type > [3]: Comparison and rejection of proposals: > https://www.dropbox.com/s/sshgtr4p30cs0vc/Python%20Callable%20Syntax%20Proposals.pdf?dl=0 > [4]: Callback protocols: > https://www.python.org/dev/peps/pep-0544/#callback-protocols > [5]: Stats on callbacks not expressible with Callable: > https://drive.google.com/file/d/1k_TqrNKcbWihRZdhMGf6K_GcLmn9ny3m/view?usp=sharing > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/ > Code of Conduct: http://python.org/psf/codeofconduct/
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4TY3MVJQOLKSTMCJRTGAZEOSIIMAWQ45/ Code of Conduct: http://python.org/psf/codeofconduct/
