Thanks for summarizing the PSC position, Barry! I agree that type annotation syntax should be consistent with the rest of the language, but I'm curious why it has to be "standard Python syntax"? Could you elaborate a bit more on the reasoning behind that constraint?
Cheers, Sergei On Fri, Oct 8, 2021 at 8:38 PM Barry Warsaw <ba...@python.org> wrote: > Thanks for thinking more deeply about how we can make type annotations for > callables more user friendly. > > The intersection between the syntax for general Python code and type > annotations has been a topic of discussion within the Steering Council (on > and off) for quite some time. We intend to provide more clarity about our > thinking, probably in the form of an informational PEP, around this topic, > and my apologies because I haven’t had time to draft it yet. > > But since this came up, I want to at least say that we are unanimous in > our belief that the type annotation language must also be standard Python > syntax. In other words, we are not in favor of letting type annotation > syntax deviate from Python syntax. As a effect of this, the PSC does not > intend to issue a blanket delegation of type-focused PEPs. Of course, > individual typing PEPs can be delegated just like any other PEP, but > especially where syntax change proposals are made, I expect that the PSC > will want keep final decision making. > > This means that any PEP which proposes syntactic changes to support typing > features must also address the implications of those syntax changes for the > general Python language. PEP 646 (Variadic Generics) is a good example of > this. Early on we recognized that the proposed syntax change had > implications for Python, and we asked the PEP authors to address those > implications for Python, which they added: > > https://www.python.org/dev/peps/pep-0646/#grammar-changes > > This is the kind of analysis the PSC needs in order to weigh the cost and > benefits of making such changes to Python. > > TL;RA - One Syntax to Rule Them All > > Cheers, > -Barry > > > On Oct 7, 2021, at 09:41, S Pradeep Kumar <gohan...@gmail.com> wrote: > > > > Hello all, > > > > Typing-sig has been discussing user-friendly syntax for the type used to > represent callables. [1] Since this affects the Python language syntax, we > wanted to get some high-level feedback from you before putting up a > detailed PEP. > > > > TL;DR: We want to propose syntax for Callable types, e.g., `(int, str) > -> bool` instead of `typing.Callable[[int, str], bool]`. Questions: 1. Are > there concerns we need to keep in mind about such a syntax change? 2. > Should we propose syntax for additional features in the same PEP or in a > future PEP? > > > > > > # Motivation > > > > Guido has pointed out that `Callable` is one of the most frequently used > type forms but has the least readable syntax. For example, say we have a > callback `formatter` that accepts a record and a list of permissions: > > > > ```python > > def print_purchases( > > user, > > formatter, # <-- callback > > ): > > <...> > > output = formatter(record, permissions) > > print(output) > > ``` > > > > To give it a type, we currently have to write: > > > > ```python > > from typing import Callable > > > > def print_purchases( > > user: User, > > formatter: Callable[[PurchaseRecord, List[AuthPermission]], > FormattedItem] # Hard to read. > > ) -> None: > > > > <...> > > output = formatter(record, permissions) > > print(output) > > ``` > > > > `Callable` can be hard to understand for new users since it doesn't > resemble existing function signatures and there can be multiple square > brackets. It also requires an import from `typing`, which is inconvenient. > Around 40% of the time [2], users just give up on precisely typing the > parameters and return type of their callbacks and just leave it as a blank > Callable. In other cases, they don't add a type for their callback at all. > Both mean that they lose the safety guarantees from typing and leave room > for bugs. > > > > We believe that adding friendly syntax for Callable types will improve > readability and encourage users to type their callbacks more precisely. > Other modern, gradually-typed languages like TypeScript (JS), Hack (PHP), > etc. have special syntax for Callable types. > > > > (As a precedent, PEP 604 recently added clean, user-friendly syntax for > the widely-used `Union` type. Instead of importing `Union` and writing > `expr: Union[AddExpr, SubtractExpr], we can just write `expr: AddExpr | > SubtractExpr`.) > > > > > > # Proposal and Questions > > > > We have a two-part proposal and questions for you: > > > > 1. Syntax to replace Callable > > > > After a lot of discussion, there is strong consensus in typing-sig about > adding syntax to replace Callable. So, the above example would be written > as: > > ```python > > def print_purchases( > > user: User, > > formatter: (PurchaseRecord, List[AuthPermission]) -> FormattedItem, > > ) -> None: > > > > <...> > > output = formatter(record, permissions) > > print(output) > > ``` > > This feels more natural and succinct. > > > > Async callbacks currently need to be written as > > ``` > > from typing import Callable > > > > async_callback: Callable[[HttpRequest], Awaitable[HttpResponse]] > > ``` > > > > With the new syntax, they would be written as > > ``` > > async_callback: async (HttpRequest) -> HttpResponse > > ``` > > which again seems more natural. There is similar syntax for the type of > decorators that pass on *args and **kwargs to the decorated function. > > > > Note that we considered and rejected using a full def-signature syntax > like > > ```` > > (record: PurchaseRecord, permissions: List[AuthPermission], /) -> > FormattedItem > > ```` > > because it would be more verbose for common cases and could lead to > subtle bugs; more details in [3]. > > > > The Callable type is also usable as an expression, like in type aliases > `IntOperator = (int, int) -> int` and `cast((int) -> int, f)` calls. > > > > **Question 1**: Are there concerns we should keep in mind about such a > syntax proposal? > > > > > > > > 2. Syntax for callback types beyond Callable > > > > `Callable` can't express the type of all possible callbacks. For > example, it doesn't support callbacks where some parameters have default > values: `formatter(record)` (the user didn't pass in `permissions`). It > *is* possible to express these advanced cases using Callback Protocols (PEP > 544) [4] but it gets verbose. > > > > There are two schools of thought on typing-sig on adding more syntax on > top of (1): > > > > (a) Some, including Guido, feel that it would be a shame to not have > syntax for core Python features like default values, keyword arguments, etc. > > > > One way to represent default values or optionally name parameters > would be: > > > > ``` > > # permissions is optional > > formatter: (PurchaseRecord, List[AuthPermission]=...) -> > FormattedItem > > > > # permissions can be called using a keyword argument. > > formatter: (PurchaseRecord, permissions: List[AuthPermission]) -> > FormattedItem > > ``` > > > > There are also alternative syntax proposals. > > > > (b) Others want to wait till we have more real-world experience with the > syntax in (1). > > > > The above cases occur < 5% of the time in typed or untyped code [5]. > And the syntax in (1) is forward-compatible with the additional proposals. > So we could add them later if needed or leave them out, since we can always > use callback protocols. > > > > **Question 2**: Do you have preferences either way? Do we propose (1) > alone or (1) + (2)? > > > > > > Once we get some high-level feedback here, we will draft a PEP going > into the details for various use cases. > > > > Best, > > Pradeep Kumar Srinivasan > > Steven Troxler > > Eric Traut > > > > PS: We've linked to more details below. Happy to provide more details as > needed. > > > > [1]: typing-sig thread about the proposal: > https://mail.python.org/archives/list/typing-...@python.org/message/JZLYRAXJV34WAV5TKEOMA32V7ZLPOBFC/ > > [2]: Stats about loosely-typed Callables: > https://github.com/pradeep90/annotation_collector#typed-projects---callable-type > > [3]: Comparison and rejection of proposals: > https://www.dropbox.com/s/sshgtr4p30cs0vc/Python%20Callable%20Syntax%20Proposals.pdf?dl=0 > > [4]: Callback protocols: > https://www.python.org/dev/peps/pep-0544/#callback-protocols > > [5]: Stats on callbacks not expressible with Callable: > https://drive.google.com/file/d/1k_TqrNKcbWihRZdhMGf6K_GcLmn9ny3m/view?usp=sharing > > _______________________________________________ > > Python-Dev mailing list -- python-dev@python.org > > To unsubscribe send an email to python-dev-le...@python.org > > https://mail.python.org/mailman3/lists/python-dev.python.org/ > > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/ > > Code of Conduct: http://python.org/psf/codeofconduct/ > > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/4TY3MVJQOLKSTMCJRTGAZEOSIIMAWQ45/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/COYIQC7QTVCACCKO6AKX7U4OS5QDROGB/ Code of Conduct: http://python.org/psf/codeofconduct/
