[Python-Dev] Re: RFC on Callable Syntax PEP

[Petr Viktorin]

On Thu, Dec 16, 2021 at 6:57 PM Steven Troxler  wrote:
>
> Hello all,
>
> Thanks everyone for comments on our earlier thread [1] about callable type 
> syntax. We now  have a draft PEP [2] proposing an arrow-based syntax for 
> callable types, for example:
>
> ```
> (int, str) -> bool # equivalent to Callable[[int, str], bool]
> ```
>
> In support of the PEP we also have:
> - a reference implementation of the parser [3] to ensure the grammar is 
> correct  (tests [5], [6], [7])
> - a detailed specification of planned runtime behavior [4], which is not yet 
> in the reference implementation
>
> We'd like to get your feedback about the PEP in general, and especially 
> details and edge cases we need to consider regarding runtime behavior.
>
> Cheers,
> Steven Troxler
>
> -
> [1] Earlier python-dev thread 
> https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/
> [2] PEP 677: https://www.python.org/dev/peps/pep-0677/
> [3] Reference implementation of Parser: 
> https://github.com/stroxler/cpython/tree/callable-type-syntax--shorthand
> [4] Details on the runtime behavior:  
> https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG71Lk/edit
>
> [5] Ast tests for parser changes:
> https://github.com/stroxler/cpython/blob/20eb59fdca0d6d8dbe4efa3b04038c7c22024654/Lib/test/test_ast.py#L359-L392
> [6] Easy-read tests of examples from the PEP: 
> https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
> [7] Test sanity checking hundreds of examples pulled from typeshed:
> https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py


Having a symbol (->) for typing when a similar expression uses a word
(lambda) seems somewhat backwards to me.
If wonder: back when lambda was added, were symbols like ->
considered? If so, how do the arguments for `lambda` vs. a symbol
compare with arguments for `Callable` vs. an arrow?

I also wonder how many of the cases would be made much more readable
by naming the callable type -- as you would do with any complex
expression -- especially if PEP 646 Variadic Generics enables you to
create specialized callable types like AsyncCallable or Decorator. For
example, this would be quite clear to me:

Customizer = AsyncCallable[[Response, list[UserSetting]], Response]
def customize_response(
response: Response,
customizer: Customizer,
) -> Response:
   ...

Naming probably won't help too much with more abstract higher-order
functions like the flat_map example. But, if `(int) -> list[int]`
makes things clear to you, Haskell might be a better choice than
Python...

---

BTW, has anyone thought about using a colon to parametrize Callable?
Callable[[Response, list[UserSetting]]: Response]
/joke
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LTZWNG2G2BEUXIBWNM4JGOEETV2BH3T6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Paul Moore]

To me, these are mostly convincing examples that people need to name
parts of a complex type like this :-)

I don't actually find any of the syntaxes better than any other. They
are all a bit bad, but I view that as the fault of the complex nested
types, not the syntax (hence my preference for naming things). It's
hard to give examples of how I'd name things, because you need to know
the "business logic" to name things well, and I don't for these.

Frankly, if people were writing normal Python expressions like this,
everyone would be telling them that Python isn't about writing
one-liners and they should factor out named subexpressions. Why should
type annotations be any different?

Paul

PS If I wanted any improvement for callables, it would be to use
parentheses for the arguments rather than square brackets (which look
too "heavy" in my view). So Callable[(int, str), bool] rather than
Callable[[int, str], bool]. I understand why the outer brackets have
to be square (abuse of indexing notation) but as far as I can see
there's no reason the inner ones need to be (OK, for single arguments,
Callable[(int,), bool] is a bit clumsy...)

On Tue, 21 Dec 2021 at 22:10, Steven Troxler  wrote:
>
> In the example I was aiming for something easy to understand that produced a 
> type illustrating potential problems of PEP 677, which is at its worst when 
> there are callables in both argument and return position. I don't have a 
> great real-world example of this worst-case, most of what I've seen involves 
> simpler and the current PEP 677 proposal isn't as bad.
>
> As for formatting I agree that I wouldn't hand-write the type as
> ```
> ((int) -> float, (str) -> bool) -> (int, str) -> tuple[float, bool]
> ```
> but a lot of code formatters might fight me on this, so I think it's worthy 
> of consideration, and I was trying to illustrate a readability problem and 
> possible fix. Formatting the code nicely so that my proposal looks good would 
> have meant not really engaging with the concern.
>
> If you're looking for examples from real code where Callable is unweildy, 
> Pradeep collected a few at [1] from typeshed although most of them look just 
> fine with the current PEP 677 proposal. A couple examples:
>
> Callable[[AnyStr, Callable[[AnyStr, AnyStr, AnyStr], AnyStr]], AnyStr]
> Callable[[Optional[str], tuple[_Marshallable, ...]], Union[Fault, 
> tuple[_Marshallable, ...]]]
> Callable[[str, Callable[[Iterable[str], str], str]], None]
>
> versus the same types written using the current PEP 677 syntax:
>
> (AnyStr, (AnyStr, AnyStr, AnyStr) -> AnyStr) -> AnyStr
> (Optional[str], tuple[_Marshallable, ...]) -> Union[Fault, 
> tuple[_Marshallable, ...]]
> (str, (Iterable[str], str) -> str) -> None
>
> versus with outer parentheses:
>
> (AnyStr, (AnyStr, AnyStr, AnyStr -> AnyStr) -> AnyStr)
> (Optional[str], tuple[_Marshallable, ...] -> Union[Fault, 
> tuple[_Marshallable, ...]])
> (str, (Iterable[str], str -> str) -> None)
>
> and another idea, requiring both outer parentheses and argument parentheses:
>
> ((AnyStr, ((AnyStr, AnyStr, AnyStr )-> AnyStr)) -> AnyStr)
> ((Optional[str], tuple[_Marshallable, ...]) -> Union[Fault, 
> tuple[_Marshallable, ...]])
> (str, (Iterable[str], str) -> str) -> None)
>
> To me, these are convincing examples of where Callable is hard to read and an 
> arrow syntax is easier, but that doesn't necessarily mean it's worth the 
> price of new syntax.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/KWQVEQDZFBZDQDLOGATXSFN7KSF4WVYJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CMDXHQPLJMGCJLL2SVPFJLC3CRE5AUYU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Steven Troxler]

In the example I was aiming for something easy to understand that produced a 
type illustrating potential problems of PEP 677, which is at its worst when 
there are callables in both argument and return position. I don't have a great 
real-world example of this worst-case, most of what I've seen involves simpler 
and the current PEP 677 proposal isn't as bad.

As for formatting I agree that I wouldn't hand-write the type as
```
((int) -> float, (str) -> bool) -> (int, str) -> tuple[float, bool]
```
but a lot of code formatters might fight me on this, so I think it's worthy of 
consideration, and I was trying to illustrate a readability problem and 
possible fix. Formatting the code nicely so that my proposal looks good would 
have meant not really engaging with the concern.

If you're looking for examples from real code where Callable is unweildy, 
Pradeep collected a few at [1] from typeshed although most of them look just 
fine with the current PEP 677 proposal. A couple examples:

Callable[[AnyStr, Callable[[AnyStr, AnyStr, AnyStr], AnyStr]], AnyStr]
Callable[[Optional[str], tuple[_Marshallable, ...]], Union[Fault, 
tuple[_Marshallable, ...]]]
Callable[[str, Callable[[Iterable[str], str], str]], None]

versus the same types written using the current PEP 677 syntax:

(AnyStr, (AnyStr, AnyStr, AnyStr) -> AnyStr) -> AnyStr
(Optional[str], tuple[_Marshallable, ...]) -> Union[Fault, tuple[_Marshallable, 
...]]
(str, (Iterable[str], str) -> str) -> None

versus with outer parentheses:

(AnyStr, (AnyStr, AnyStr, AnyStr -> AnyStr) -> AnyStr)
(Optional[str], tuple[_Marshallable, ...] -> Union[Fault, tuple[_Marshallable, 
...]])
(str, (Iterable[str], str -> str) -> None)

and another idea, requiring both outer parentheses and argument parentheses:

((AnyStr, ((AnyStr, AnyStr, AnyStr )-> AnyStr)) -> AnyStr)
((Optional[str], tuple[_Marshallable, ...]) -> Union[Fault, 
tuple[_Marshallable, ...]])
(str, (Iterable[str], str) -> str) -> None)

To me, these are convincing examples of where Callable is hard to read and an 
arrow syntax is easier, but that doesn't necessarily mean it's worth the price 
of new syntax.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KWQVEQDZFBZDQDLOGATXSFN7KSF4WVYJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Gregory Beauregard via Python-Dev]

Ahah, that makes sense. But then I think I buy into your reasoning that this 
isn't really a replacement for callable syntax (although functions as types 
opens up other interesting possibilities). For this and other reasons I'd hate 
to see callable syntax rejected in favor of it, so definite +1 on new callable 
syntax for me.

p.s. I'm +0.5 on | binding tighter than ->
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZURCWPSOE72MFPWONUIOH3XWVJ6ETWR5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Ken Jin]

I agree with what Batuhan said. Adding on, I'm very concerned about the 
potential maintenance burden for Python implementations.

Just for typing in CPython, some combination of the following knowledge is 
required to contribute code:
1. Metaclasses
2. Descriptors and CPython dunders
3. C internals due to PEP 585 and 604
4. If we throw in annotations internals, maybe we need some parser knowledge 
too, but this is usually not required.

With the new callable syntax, that likely needs strong parser and compiler 
knowledge. I'm not claiming everyone needs to know all of the above to 
contribute (to the contrary, not knowing most of these is fine), but IMO the 
list is getting very long. I'd be sad if one day new typing contributors feel 
too overwhelmed by typing's implementation complexity (I admit sometimes 
feeling overwhelmed too).

That said, thank you PEP authors for working to make Python's typing syntax 
more elegant. I hope my concerns didn't come across as overly harsh.

- KJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YZG53T2D3UO6WZCR5U3N45GVAK3E7JPN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Paul Moore]

On Tue, 21 Dec 2021 at 18:35, Steven Troxler  wrote:
>
> I've been thinking about readability hard because I share many of your 
> concerns about readability.

Before I comment on syntax, I'd like to question the example:

> An example
> =
>
> The function
> 
>
> To get at what I mean, here's a nice simple function:
> ```
> def zip(f, g):
> def zipped(x: int, y: str):
> return f(x), g(y)
> return zipped
> ```

So my first question is why a generic function like this would limit
the parameters x and y to int and str. Surely this is an entirely
workable function for *any* arguments with the appropriate "shape"? So
how is this a good example of somewhere you'd use types like
Callable[[int], bool] (by the way, where did that "bool" appear from?)
in a signature?

My immediate thought when seeing that zip function is that its type is
"obvious"¹, (takes two functions, returns a function which takes 2
arguments matching the args of the 2 functions and returns a tuple
matching the results of the 2 functions). That type is almost
impossible to express clearly, because it needs pretty complex
generics. But the more important point is that I'd never, **ever**,
want to write that type out longhand. I'd expect a type system to
either infer the type, and it would be anonymous¹, or to give up and
treat it as untyped. One thing I absolutely would not want to do is
over-constrain any of the arguments just to make it possible to write
the type down.

The real issue with this function, in my view, is not expressing
callables, but rather generic type variables ("return a function whose
first argument has the same type as the single argument of the
function which is the first argument of this function..." !!!)

Can you suggest a more "real world" function as an example, which
focuses on the callable syntax and doesn't use things like arguments
called f and g? Maybe a GUI callback with a function argument like
on_click?

¹ By which I mean intuitive, not easy to express in words!!! :-)
² Some languages may allow syntax like `typeof(zip)` to refer to that
anonymous type, but that's a separate point.

> Here's it's type using typing.Callable:
> ```
> typing.Callable[
> [typing.Callable[[int], bool], typing.Callable[[str], float]],
> typing.Callable[[int, str], tuple[bool, float]
> ]
> ```
> which seems ugly. It's actually not bad compared to a lot of production code, 
> but this is the kind of thing that led to PEP 677.

It's massively over-constrained. I assume that's because you're trying
to make a point about Callable[] rather than about generics, but can
you give a realistic example that *doesn't* involve over-constraining
higher order functions?

On a side note, why not name at least some of those function types?
And why not use "from typing import Callable"? It feels like you're
not making enough effort to make your example readable, which
undermines your point as a result.

> ((int) -> float, (str) -> bool) -> (int, str) -> tuple[float, bool]

To your credit, you've made this pretty unreadable, which gives some
balance here :-) Seriously, making it a one-liner with all those ->
arrows is a disaster. Changing the location of the parentheses doesn't
alter that at all.

Rewriting as a multi-line expression:

(
(int) -> float,
(str) -> bool
) -> (int, str) -> tuple[float, bool]

helps quite a bit, but returning a function looks bad here. We're not
writing Haskell, you know ;-)

I'd prefer a "mixed" notation here:

(
(int) -> float,
(str) -> bool
) -> Callable[(int, str), tuple[float, bool]]

I don't honestly think there's a readable "function returning a
function" form here - the chained -> tokens is just awkward. Although
that's clearly a matter of preference, there's never going to be an
objective answer here.

> Here’s the type if we change the syntax to put the right parenthesis after 
> the return type:
> ```
> ((int -> float), (str -> bool) -> (int, str -> tuple[float, bool])
> ```
>
> To my eyes, most of the pain points are now eliminated.
> - there’s never a double-arrow due to callable in return position
> - even for argument types, to my eyes it’s now easier to read

To my mind, the eye is still drawn to the arrows, and the readability
is no better. And the parentheses give me a lisp vibe, for reasons I
can't really pin down but which makes this version *less* readable.

> An added bonus is we no longer have to think about double-arrows when a 
> callable type is in the return position of a function, e.g.:
> ```
> def f() -> (int, str -> bool): ...
> ```

Still looks like double arrows to me, I'm afraid. The parentheses
don't group strongly enough to override the "chain of arrows"
impression.

> And it solves another major usability problem we found - writing optional 
> callables - because it’s now no problem at all to write
> ```
> (int, str -> bool) | None
> ```
> as the type of an optional callable argument.

I guess, but it feels like punctuation 

[Python-Dev] Re: RFC on Callable Syntax PEP

[Steven Troxler]

I've been thinking about readability hard because I share many of your concerns 
about readability.

I'm starting to think parenthesizing the outside might work much better in 
practice:
```
(int, str -> bool)
```
instead of
```
(int, str) -> bool
```

This looks a bit less familiar, but it
- eliminates any situation
- helps the eye group together functions (and lets us use an editor's 
go-to-delimiter if we get lost!)


An example
=

The function


To get at what I mean, here's a nice simple function:
```
def zip(f, g):
def zipped(x: int, y: str):
return f(x), g(y)
return zipped
```



typing.Callable


Here's it's type using typing.Callable:
```
typing.Callable[
[typing.Callable[[int], bool], typing.Callable[[str], float]],
typing.Callable[[int, str], tuple[bool, float]
]
```
which seems ugly. It's actually not bad compared to a lot of production code, 
but this is the kind of thing that led to PEP 677.


current PEP 677 proposal
--


With the current PEP 677 proposal, writing the type on one line we get
```
((int) -> float, (str) -> bool) -> (int, str) -> tuple[float, bool]
```

I prefer this to the Callable version. But it’s dense, and I do agree it 
illustrates some readability issues with PEP 677.



Using (int, str -> bool) syntax

Here’s the type if we change the syntax to put the right parenthesis after the 
return type:
```
((int -> float), (str -> bool) -> (int, str -> tuple[float, bool])
```

To my eyes, most of the pain points are now eliminated.
- there’s never a double-arrow due to callable in return position
- even for argument types, to my eyes it’s now easier to read

An added bonus is we no longer have to think about double-arrows when a 
callable type is in the return position of a function, e.g.:
```
def f() -> (int, str -> bool): ...
```

And it solves another major usability problem we found - writing optional 
callables - because it’s now no problem at all to write
```
(int, str -> bool) | None
```
as the type of an optional callable argument.


New edge cases / readability issues?
-

I’m pretty sure *all* existing edge cases and readability issues are basically 
solved by this syntax because the grouping becomes obvious.

The only new edge case I can see is that we might not like `(-> bool)` for 
parameterless callables. I’d be okay with that, but I’d also be down with a 
hack like `(() -> bool)`.

I don’t actually think it matters much - by making this change we would trade 
multiple edge cases in *complicated* examples for a single edge case in the 
*simplest possible* example where readability isn’t a big problem.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AXRKGCPP2GTDRINQ6NQYM743Y6EH6FO5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Steven Troxler]

In most of our discussions of this idea, we've assumed that we'd adopt the same 
semantics that callback protocols use.

If we do that, then only `lambda a: 3` will type check. In order to type check 
both you'd have to make `a` positional-only:
```
def IntToIntFunc(a: int, /) -> int:
...
```

This is one of the reasons I think functions-as-types could be a great idea but 
not a good substitute for better callable syntax.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6EB54EZXUVFR3HPTGLQK4AGJ7UUR5K4F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Mehdi2277]

I don’t see how this helps much in situations with nested callables which are 
very common for decorators. I’m also unsure how this will work with situations 
where input and output are both callables and the signature is modified using 
paramspec. An example would be a decorator that adds/removes one argument. Is 
Concatenate[int, InttoIntFunction] a thing?

I also would ideally want a very light weight way to handle simple callables 
like (int) -> bool. This decorator solution of @Callable seems roughly as 
readable as existing solution of using callback protocols or a type alias.

I took a quick scan of my team’s codebase and see about 150 callable vs like 
1.5k optional. Optional is fairly high as a lot of our code has many arguments 
that can be disabled which I think is normal for ml/numerical libraries with 
tons of tuning settings. The most complex callable signatures tend to be 
decorators/decorator factories for us.

Main place I find this useful is skimming other libraries type signatures and 
hovering in ides. Nice hover type hints is helpful over large blocks of 
callable brackets.

I do find this syntactic sugar nice. I’m +0.5, mainly as while I think current 
callable generic is messy, I also think most of the people that write those 
type annotations are the ones that write a lot of them. Maybe that would change 
with better readability but right now heavy callable type usage feel like an 
advanced, uncommon thing. Part of it is also heavy functional style code isn’t 
that common in python.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EVE274LSWE5CDPJYW3VVVPJENVQIQMVM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Gregory Beauregard via Python-Dev]

Does `lambda b: 3` type check with `IntToIntFunc` or only `lambda a: 3`? The 
intent seems to be it's more like `Callable` (where the argument name is not 
important), but maybe both could be supported? I wonder about making more use 
of the `_` soft keyword where calling a function with multiple `_` is a runtime 
error. Maybe it would make too much of a mess.

After some testing evidently mypy only applies its knowledge sometimes anyway: 
https://github.com/python/mypy/issues/11807
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MDR2TCLEN7YKNUHKID5D6KMBJL73UBOZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Patrick Reader]

On 20/12/2021 22:34, Serhiy Storchaka wrote:
> 20.12.21 21:28, Brett Cannon пише:
>> As someone with use of this, would you find this useful (i.e. +1, +0)?
>> Serhiy already said "no" in another thread.
> In every file we import 5-10 or more names from the typing module. We
> still does not use PEP 585 and PEP 604 syntax, but are going to do this
> in near future. It could save as from importing List, Tuple, Dict, Set,
> Union, Optional, but we still need to import Any, Sequence, Iterable,
> Iterator, AsyncIterator, Awaitable, TypeVar, and sometimes
> AsyncContextManager, NewType, cast, overload. Special syntax for
> callable type hints will not save game.
Sequence, Iterable, Iterator, AsyncIterator, Awaitable, AsyncContextManager are 
all in collections.abc not typing now. Of course it doesn't reduce total 
imports but it makes it less likely that you'll need to import typing.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PVF4G57RJ634QYFJ2HGI3QQ5RXB6J455/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Guido van Rossum]

Yeah, making the body optional (without looking at decorators) is not
acceptable either. Too easy to do by mistake (I still do this All. The.
Time. :-)

On Mon, Dec 20, 2021 at 2:19 PM  wrote:

> My proposal wasn't to make the body optional based on the presence of a
> decorator, but rather to return a "function prototype" iff the body does
> not exist (I probably should have made my made my own reply instead of
> piggybacking on his proposal). I also mentioned some form of expression to
> represent this,  similar to lambda. Maybe a friendly error message telling
> the user to use a function when this thing is called would alleviate some
> confusion? I'm not sure how one would forget to add the function body
> anyway.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/3HJY6VGLIWRJ7F4BZBNGNCSJJTXH3CVM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6OCPFRM5UCRTUOZ2KFMG6ZV4XMS3YNJD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Steven Troxler]

Thanks for the feedback. I have a few thoughts.


(1) Concerns about complexity of the syntax make sense to me, it's definitely 
possible to write confusing types with this syntax. Readability would be a good 
reason to reject this idea, but it does cut both ways because `Callable` can be 
hard to read today.

Typing-sig as a whole is confident that an arrow syntax would be an improvement 
but I think the submission process is a good time for wider opinions, I can see 
that it might not be a good idea overall even if it's a better type syntax.


(2) I do like the idea of allowing parameters in a tuple in the existing 
Callable type, that seems like a very clear easy win if we *don't* accept 
callable syntax. I agree that tuple-for-parameters combined with making 
`builtins.callable` subscriptable would address a chunk of the concerns.



I can look into getting more stats on async callables, I don't think they are 
terribly common in most projects, but in certain contexts like async webservers 
they can come up quite a lot.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BXX3IHTGXCBBL6XKSEGK366CFGFDNLHX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Serhiy Storchaka]

20.12.21 21:28, Brett Cannon пише:
> As someone with use of this, would you find this useful (i.e. +1, +0)?
> Serhiy already said "no" in another thread.

In every file we import 5-10 or more names from the typing module. We
still does not use PEP 585 and PEP 604 syntax, but are going to do this
in near future. It could save as from importing List, Tuple, Dict, Set,
Union, Optional, but we still need to import Any, Sequence, Iterable,
Iterator, AsyncIterator, Awaitable, TypeVar, and sometimes
AsyncContextManager, NewType, cast, overload. Special syntax for
callable type hints will not save game.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NLLN4IGOKCQZP7LYM7VBQSGGGWPW3WKZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[asleep . cult]

My proposal wasn't to make the body optional based on the presence of a 
decorator, but rather to return a "function prototype" iff the body does not 
exist (I probably should have made my made my own reply instead of piggybacking 
on his proposal). I also mentioned some form of expression to represent this,  
similar to lambda. Maybe a friendly error message telling the user to use a 
function when this thing is called would alleviate some confusion? I'm not sure 
how one would forget to add the function body anyway.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3HJY6VGLIWRJ7F4BZBNGNCSJJTXH3CVM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Guido van Rossum]

On Mon, Dec 20, 2021 at 12:44 PM  wrote:

> This is such a great idea that I think it deserves its own PEP (to compete
> with this one?) Let me explain. PEP 677 was created for the sole purpose of
> replacing typing.Callable, but there are still some other areas where
> function metadata is required. What if we instead introduced a function
> prototype that allows you to "declare" a "function" without a body.
>
> typing example:
>
> import typing
>
> @typing.Callable
> def IntToIntFunc(a: int) -> int
>
> def flat_map(
> l: list[int],
> func: IntToIntFunc
> ) -> list[int]:
> ...
>
> ctypes example:
>
> import ctypes
>
> @ctypes.CFUNCTYPE
> def f(x: int) -> bool
>
> But of course this comes with a few issues: should it be an expression and
> if so, should the name be optional? How can ParamSpec be handled?
>

Allowing 'def' without a body based on the presence or absence of a
decorator sounds like it will just confuse people and cause bizarre errors
if people accidentally leave out a body. Let's not go there.

However, Lukasz has already proposed a very similar mechanism (with dummy
body), without needing a decorator. His proposal is simply:

def IntToIntFunc(a: int) -> int: ...

def flat_map(l: list[int], func: IntToIntFunc) -> list[int]:
# body

No need for a `@Callable` decorator!

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/64RWUSM5IR7ZLDPGIMVWEWXC4GVKUPW4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[asleep . cult]

This is such a great idea that I think it deserves its own PEP (to compete with 
this one?) Let me explain. PEP 677 was created for the sole purpose of 
replacing typing.Callable, but there are still some other areas where function 
metadata is required. What if we instead introduced a function prototype that 
allows you to "declare" a "function" without a body.

typing example:

import typing

@typing.Callable
def IntToIntFunc(a: int) -> int

def flat_map(
l: list[int],
func: IntToIntFunc
) -> list[int]:
...

ctypes example:

import ctypes

@ctypes.CFUNCTYPE
def f(x: int) -> bool

But of course this comes with a few issues: should it be an expression and if 
so, should the name be optional? How can ParamSpec be handled?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XPPZZZWVVW6KRBKYXJKXHPTECRDIOFUE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Brett Cannon]

As someone with use of this, would you find this useful (i.e. +1, +0)?
Serhiy already said "no" in another thread.

On Mon, Dec 20, 2021 at 4:38 AM Andrew Svetlov 
wrote:

> Perhaps Serhiy did more accurate counting, my estimate is very rough.
>
> On Mon, Dec 20, 2021 at 2:15 PM Serhiy Storchaka 
> wrote:
>
>> 20.12.21 13:42, Mark Shannon пише:
>> > OOI, of those 1577 Callable type hints, how many distinct Callable
>> types?
>>
>> Around 15-20%. Most of them are in tests which widely use pytest
>> fixtures, so functions taking and returning callables are common. There
>> are around 200 occurrences in non-test code, half of them are distinct
>> types.
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Thanks,
> Andrew Svetlov
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/Q4A4ZTJHSS6ALDXHZJEERJBVGJNYC6KV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2OPXHGWUDCFTDIYEARIRJFIGWLM6JTLR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Brett Cannon]

On Mon, Dec 20, 2021 at 3:44 AM Mark Shannon  wrote:

> Hi,
>
> Why not make Callable usable as a function decorator?
>
>
>
> The motivating example in the PEP is this:
>
>
> def flat_map(
>  l: list[int],
>  func: Callable[[int], list[int]]
> ) -> list[int]:
>  
>
>
> Since, as the PEP claims, `Callable[[int], list[int]]` is hard to read,
> then give it a name and use regular function definition syntax.
>
>
> @Callable
> def IntToIntFunc(a:int)->int:
>  pass
>
>
> def flat_map(
>  l: list[int],
>  func: IntToIntFunc
> ) -> list[int]:
>  
>
>
> To me, this seems much clearer than the proposed syntax and is more
> general.
>
>
> It is a little longer, but unless you are playing code golf, that
> shouldn't matter.
>

It's an interesting idea! Both `@overload` and `@final` show there is
precedence for having decorators have special meanings to static type
checkers:
https://docs.python.org/3/library/typing.html#functions-and-decorators.


>
> Cheers,
> Mark.
>
>
>
> On 16/12/2021 5:57 pm, Steven Troxler wrote:
> > Hello all,
> >
> > Thanks everyone for comments on our earlier thread [1] about callable
> type syntax. We now  have a draft PEP [2] proposing an arrow-based syntax
> for callable types, for example:
> >
> > ```
> > (int, str) -> bool # equivalent to Callable[[int, str], bool]
> > ```
> >
> > In support of the PEP we also have:
> > - a reference implementation of the parser [3] to ensure the grammar is
> correct  (tests [5], [6], [7])
> > - a detailed specification of planned runtime behavior [4], which is not
> yet in the reference implementation
> >
> > We'd like to get your feedback about the PEP in general, and especially
> details and edge cases we need to consider regarding runtime behavior.
> >
> > Cheers,
> > Steven Troxler
> >
> > -
> > [1] Earlier python-dev thread
> https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/
> > [2] PEP 677: https://www.python.org/dev/peps/pep-0677/
> > [3] Reference implementation of Parser:
> https://github.com/stroxler/cpython/tree/callable-type-syntax--shorthand
> > [4] Details on the runtime behavior:
> https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG71Lk/edit
> >
> > [5] Ast tests for parser changes:
> >
> https://github.com/stroxler/cpython/blob/20eb59fdca0d6d8dbe4efa3b04038c7c22024654/Lib/test/test_ast.py#L359-L392
> > [6] Easy-read tests of examples from the PEP:
> https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
> > [7] Test sanity checking hundreds of examples pulled from typeshed:
> >
> https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OGACYN2X7RX2GHAUP2AKRPT6DP432VCN/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2BKD5YBU7WJMUY3TSX34HX5IICT5UFRQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OCLGF6IXGHVBCCA24ZFJMUEFJ2DJVAQX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Brett Cannon]

On Sun, Dec 19, 2021 at 8:26 PM Christopher Barker 
wrote:

>
> On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:
>>
>> > tl;dr: I find it very troubling that we are going on a path where need
>> > to increase the language complexity (syntax) only in the cause
>> > 'easier' typing.
>
>
> Which brings up the question is whether it's worth adding  syntax for
> typing, but only in the context of typing.
>

The SC has already said we don't like that idea as that bifurcates the
knowledge one needs in order to even have a chance at comprehending a type
hint. Plus PEP 649 wouldn't be possible in that instance unless we ship a
second parser just for type hints in order to translate the type-specific
syntax to type-related objects.

-Brett


> As of right now, typing.get_type_hints() will evaluate a string
> annotation, e.g.
>
> In [62]: def f(x:"int"):
> ...: pass
> ...:
>
> In [63]: typing.get_type_hints(f)
> Out[63]: {'x': int}
>
> so get_type_hints could extend its acceptable syntax with this new use of
> -> -- and it could get used by wrapping it in quotes. And depending on
> how PEP 563 gets resolved, the quotes may not be necessary in the future.
>
> And this could open up some other nifty things, like extending what's
> allowable inside [] -- there was a discussion a while back on python-ideas
> about extending the __getitem__ protocol, partly motivated by type hints.
>
> -CHB
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JWPMHLSPFTYAYSMWCX7LOXWBEU4FA6DC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N7HF4FDSPEPJDWFD56NFUPHOEBU4AS7D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Andrew Svetlov]

Perhaps Serhiy did more accurate counting, my estimate is very rough.

On Mon, Dec 20, 2021 at 2:15 PM Serhiy Storchaka 
wrote:

> 20.12.21 13:42, Mark Shannon пише:
> > OOI, of those 1577 Callable type hints, how many distinct Callable types?
>
> Around 15-20%. Most of them are in tests which widely use pytest
> fixtures, so functions taking and returning callables are common. There
> are around 200 occurrences in non-test code, half of them are distinct
> types.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q4A4ZTJHSS6ALDXHZJEERJBVGJNYC6KV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Serhiy Storchaka]

20.12.21 13:42, Mark Shannon пише:
> OOI, of those 1577 Callable type hints, how many distinct Callable types?

Around 15-20%. Most of them are in tests which widely use pytest
fixtures, so functions taking and returning callables are common. There
are around 200 occurrences in non-test code, half of them are distinct
types.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7YXSPACKOU7AGOHEZX2VAMXEELA3OZGA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Andrew Svetlov]

On Mon, Dec 20, 2021 at 1:42 PM Mark Shannon  wrote:

> OOI, of those 1577 Callable type hints, how many distinct Callable types?
>
>
Good question. About 30 callables for source code itself and an additional
60 for pytest factory fixtures.


> On 20/12/2021 7:52 am, Andrew Svetlov wrote:
> > At my job, we have 1577 Callable type hints scattered in 1063 Python
> files.
> > For comparison, this codebase also has 2754 dict annotations and 1835
> list ones.
> >
> > On Mon, Dec 20, 2021 at 8:11 AM Christopher Barker  > wrote:
> >
> > note: I wasn't thinking -- typeshed, of course, has a lot more than
> the standard lib.  But it's still a collection of widely used somewhat
> general purpose libraries. So I think my hypothesis is still valid.
> >
> > -CHB
> >
> >
> > On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker <
> python...@gmail.com > wrote:
> >
> > A question that came up for me is:
> >
> > How common is it to need to use Callable for type hints?
> particularly complex versions, specifying what parameters the Callable
> takes? A more compact and easier to read syntax is nice, but not very
> important if it isn't used much.
> >
> > My first thought on this was that I can't remember a single time
> that I wrote production code that took a Callable as a function parameter
> -- or returned one -- OK maybe a few times, but it's certainly rare in my
> production code.
> >
> > So I looked in the PEP to see if that issue was addressed, and
> indeed it is:
> >
> > "The Callable type is widely used. For example, as of October
> 2021 it was the fifth most common complex type in typeshed,"
> >
> > That did surprise me, but on thinking about it, maybe not so
> much. It strikes me that Callable is most likely to be used in fairly low
> level, general purpose functions, like map(), sort(), various functions in
> itertools, etc. Just the sort of functions that are common in the standard
> library, but may not so much in production code.
> >
> > I have no idea how to evaluate how common it is in production
> code -- maybe type hinting is common enough now  that PyPi could be
> searched -- but even PyPi is a biased sample, as it is full of, by
> definition, libraries for others' use -- i.e. general purpose tools (less
> general that the stad lib, but still not specialty production code, which I
> suspect is the majority of Python code out there).
> >
> > Perhaps some folks that have been type=hinting their production
> code bases could provide anecdotal evidence.
> >
> > Anyway, if my hypothesis is correct, then it's not so bad that
> not-so-nice syntax is required to type hint general purpose utilities.
> >
> > -CHB
> >
> > --
> > Christopher Barker, PhD (Chris)
> >
> > Python Language Consulting
> >- Teaching
> >- Scientific Software Development
> >- Desktop GUI and Web Development
> >- wxPython, numpy, scipy, Cython
> >
> >
> >
> > --
> > Christopher Barker, PhD (Chris)
> >
> > Python Language Consulting
> >- Teaching
> >- Scientific Software Development
> >- Desktop GUI and Web Development
> >- wxPython, numpy, scipy, Cython
> > ___
> > Python-Dev mailing list -- python-dev@python.org  python-dev@python.org>
> > To unsubscribe send an email to python-dev-le...@python.org  python-dev-le...@python.org>
> > https://mail.python.org/mailman3/lists/python-dev.python.org/ <
> https://mail.python.org/mailman3/lists/python-dev.python.org/>
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
> <
> https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
> >
> > Code of Conduct: http://python.org/psf/codeofconduct/ <
> http://python.org/psf/codeofconduct/>
> >
> >
> >
> > --
> > Thanks,
> > Andrew Svetlov
> >
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JYZGIDEBV4R5E7XXT3KFS2O545TDTAGT/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MG2PKYBIZHA3PPEVWJMSXR4WJ4TLBKVZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Mark Shannon]

OOI, of those 1577 Callable type hints, how many distinct Callable types?


On 20/12/2021 7:52 am, Andrew Svetlov wrote:

At my job, we have 1577 Callable type hints scattered in 1063 Python files.
For comparison, this codebase also has 2754 dict annotations and 1835 list ones.

On Mon, Dec 20, 2021 at 8:11 AM Christopher Barker mailto:python...@gmail.com>> wrote:

note: I wasn't thinking -- typeshed, of course, has a lot more than the 
standard lib.  But it's still a collection of widely used somewhat general 
purpose libraries. So I think my hypothesis is still valid.

-CHB


On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker mailto:python...@gmail.com>> wrote:

A question that came up for me is:

How common is it to need to use Callable for type hints? particularly 
complex versions, specifying what parameters the Callable takes? A more compact 
and easier to read syntax is nice, but not very important if it isn't used much.

My first thought on this was that I can't remember a single time that I 
wrote production code that took a Callable as a function parameter -- or 
returned one -- OK maybe a few times, but it's certainly rare in my production 
code.

So I looked in the PEP to see if that issue was addressed, and indeed 
it is:

"The Callable type is widely used. For example, as of October 2021 it was 
the fifth most common complex type in typeshed,"

That did surprise me, but on thinking about it, maybe not so much. It 
strikes me that Callable is most likely to be used in fairly low level, general 
purpose functions, like map(), sort(), various functions in itertools, etc. 
Just the sort of functions that are common in the standard library, but may not 
so much in production code.

I have no idea how to evaluate how common it is in production code -- 
maybe type hinting is common enough now  that PyPi could be searched -- but 
even PyPi is a biased sample, as it is full of, by definition, libraries for 
others' use -- i.e. general purpose tools (less general that the stad lib, but 
still not specialty production code, which I suspect is the majority of Python 
code out there).

Perhaps some folks that have been type=hinting their production code 
bases could provide anecdotal evidence.

Anyway, if my hypothesis is correct, then it's not so bad that 
not-so-nice syntax is required to type hint general purpose utilities.

-CHB

-- 
Christopher Barker, PhD (Chris)


Python Language Consulting
   - Teaching
   - Scientific Software Development
   - Desktop GUI and Web Development
   - wxPython, numpy, scipy, Cython



-- 
Christopher Barker, PhD (Chris)


Python Language Consulting
   - Teaching
   - Scientific Software Development
   - Desktop GUI and Web Development
   - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org 

To unsubscribe send an email to python-dev-le...@python.org 

https://mail.python.org/mailman3/lists/python-dev.python.org/ 

Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
 

Code of Conduct: http://python.org/psf/codeofconduct/ 




--
Thanks,
Andrew Svetlov

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JYZGIDEBV4R5E7XXT3KFS2O545TDTAGT/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RQ32VDXY7KUXHCUM6DASPVZ434PN5GLM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Mark Shannon]

Hi,

Why not make Callable usable as a function decorator?



The motivating example in the PEP is this:


def flat_map(
l: list[int],
func: Callable[[int], list[int]]
) -> list[int]:



Since, as the PEP claims, `Callable[[int], list[int]]` is hard to read, then 
give it a name and use regular function definition syntax.


@Callable
def IntToIntFunc(a:int)->int:
pass


def flat_map(
l: list[int],
func: IntToIntFunc
) -> list[int]:



To me, this seems much clearer than the proposed syntax and is more general.


It is a little longer, but unless you are playing code golf, that shouldn't 
matter.

Cheers,
Mark.



On 16/12/2021 5:57 pm, Steven Troxler wrote:

Hello all,

Thanks everyone for comments on our earlier thread [1] about callable type 
syntax. We now  have a draft PEP [2] proposing an arrow-based syntax for 
callable types, for example:

```
(int, str) -> bool # equivalent to Callable[[int, str], bool]
```

In support of the PEP we also have:
- a reference implementation of the parser [3] to ensure the grammar is correct 
 (tests [5], [6], [7])
- a detailed specification of planned runtime behavior [4], which is not yet in 
the reference implementation

We'd like to get your feedback about the PEP in general, and especially details 
and edge cases we need to consider regarding runtime behavior.

Cheers,
Steven Troxler

-
[1] Earlier python-dev thread 
https://mail.python.org/archives/list/python-dev@python.org/thread/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/
[2] PEP 677: https://www.python.org/dev/peps/pep-0677/
[3] Reference implementation of Parser: 
https://github.com/stroxler/cpython/tree/callable-type-syntax--shorthand
[4] Details on the runtime behavior:  
https://docs.google.com/document/d/15nmTDA_39Lo-EULQQwdwYx_Q1IYX4dD5WPnHbFG71Lk/edit

[5] Ast tests for parser changes:
https://github.com/stroxler/cpython/blob/20eb59fdca0d6d8dbe4efa3b04038c7c22024654/Lib/test/test_ast.py#L359-L392
[6] Easy-read tests of examples from the PEP: 
https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
[7] Test sanity checking hundreds of examples pulled from typeshed:
https://github.com/stroxler/cpython/blob/callable-type-syntax--shorthand/Lib/test/test_callable_type_examples_for_pep.py
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OGACYN2X7RX2GHAUP2AKRPT6DP432VCN/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2BKD5YBU7WJMUY3TSX34HX5IICT5UFRQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Serhiy Storchaka]

18.12.21 23:07, Terry Reedy пише:
> Batuhan expresses my concerns better than I could, so I just add my
> agreement.
> 
> On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:
> 
>> tl;dr: I find it very troubling that we are going on a path where need
>> to increase the language complexity (syntax) only in the cause
>> 'easier' typing. So I am opposed to this change.

I concur.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DVREYSY6BIPCESCOT4QP2SV5CPG53LXB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Andrew Svetlov]

At my job, we have 1577 Callable type hints scattered in 1063 Python files.
For comparison, this codebase also has 2754 dict annotations and 1835 list
ones.

On Mon, Dec 20, 2021 at 8:11 AM Christopher Barker 
wrote:

> note: I wasn't thinking -- typeshed, of course, has a lot more than the
> standard lib.  But it's still a collection of widely used somewhat general
> purpose libraries. So I think my hypothesis is still valid.
>
> -CHB
>
>
> On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker 
> wrote:
>
>> A question that came up for me is:
>>
>> How common is it to need to use Callable for type hints? particularly
>> complex versions, specifying what parameters the Callable takes? A more
>> compact and easier to read syntax is nice, but not very important if it
>> isn't used much.
>>
>> My first thought on this was that I can't remember a single time that I
>> wrote production code that took a Callable as a function parameter -- or
>> returned one -- OK maybe a few times, but it's certainly rare in my
>> production code.
>>
>> So I looked in the PEP to see if that issue was addressed, and indeed it
>> is:
>>
>> "The Callable type is widely used. For example, as of October 2021 it was
>> the fifth most common complex type in typeshed,"
>>
>> That did surprise me, but on thinking about it, maybe not so much. It
>> strikes me that Callable is most likely to be used in fairly low level,
>> general purpose functions, like map(), sort(), various functions in
>> itertools, etc. Just the sort of functions that are common in the standard
>> library, but may not so much in production code.
>>
>> I have no idea how to evaluate how common it is in production code --
>> maybe type hinting is common enough now  that PyPi could be searched -- but
>> even PyPi is a biased sample, as it is full of, by definition,
>> libraries for others' use -- i.e. general purpose tools (less general that
>> the stad lib, but still not specialty production code, which I suspect is
>> the majority of Python code out there).
>>
>> Perhaps some folks that have been type=hinting their production code
>> bases could provide anecdotal evidence.
>>
>> Anyway, if my hypothesis is correct, then it's not so bad that
>> not-so-nice syntax is required to type hint general purpose utilities.
>>
>> -CHB
>>
>> --
>> Christopher Barker, PhD (Chris)
>>
>> Python Language Consulting
>>   - Teaching
>>   - Scientific Software Development
>>   - Desktop GUI and Web Development
>>   - wxPython, numpy, scipy, Cython
>>
>
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JYZGIDEBV4R5E7XXT3KFS2O545TDTAGT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Christopher Barker]

note: I wasn't thinking -- typeshed, of course, has a lot more than the
standard lib.  But it's still a collection of widely used somewhat general
purpose libraries. So I think my hypothesis is still valid.

-CHB


On Sun, Dec 19, 2021 at 8:54 PM Christopher Barker 
wrote:

> A question that came up for me is:
>
> How common is it to need to use Callable for type hints? particularly
> complex versions, specifying what parameters the Callable takes? A more
> compact and easier to read syntax is nice, but not very important if it
> isn't used much.
>
> My first thought on this was that I can't remember a single time that I
> wrote production code that took a Callable as a function parameter -- or
> returned one -- OK maybe a few times, but it's certainly rare in my
> production code.
>
> So I looked in the PEP to see if that issue was addressed, and indeed it
> is:
>
> "The Callable type is widely used. For example, as of October 2021 it was
> the fifth most common complex type in typeshed,"
>
> That did surprise me, but on thinking about it, maybe not so much. It
> strikes me that Callable is most likely to be used in fairly low level,
> general purpose functions, like map(), sort(), various functions in
> itertools, etc. Just the sort of functions that are common in the standard
> library, but may not so much in production code.
>
> I have no idea how to evaluate how common it is in production code --
> maybe type hinting is common enough now  that PyPi could be searched -- but
> even PyPi is a biased sample, as it is full of, by definition,
> libraries for others' use -- i.e. general purpose tools (less general that
> the stad lib, but still not specialty production code, which I suspect is
> the majority of Python code out there).
>
> Perhaps some folks that have been type=hinting their production code bases
> could provide anecdotal evidence.
>
> Anyway, if my hypothesis is correct, then it's not so bad that not-so-nice
> syntax is required to type hint general purpose utilities.
>
> -CHB
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
>


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LWIXFDUGRM6Z3KHI3YGV65HWXRD2S4H5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Christopher Barker]

A question that came up for me is:

How common is it to need to use Callable for type hints? particularly
complex versions, specifying what parameters the Callable takes? A more
compact and easier to read syntax is nice, but not very important if it
isn't used much.

My first thought on this was that I can't remember a single time that I
wrote production code that took a Callable as a function parameter -- or
returned one -- OK maybe a few times, but it's certainly rare in my
production code.

So I looked in the PEP to see if that issue was addressed, and indeed it is:

"The Callable type is widely used. For example, as of October 2021 it was
the fifth most common complex type in typeshed,"

That did surprise me, but on thinking about it, maybe not so much. It
strikes me that Callable is most likely to be used in fairly low level,
general purpose functions, like map(), sort(), various functions in
itertools, etc. Just the sort of functions that are common in the standard
library, but may not so much in production code.

I have no idea how to evaluate how common it is in production code -- maybe
type hinting is common enough now  that PyPi could be searched -- but even
PyPi is a biased sample, as it is full of, by definition, libraries for
others' use -- i.e. general purpose tools (less general that the stad lib,
but still not specialty production code, which I suspect is the majority of
Python code out there).

Perhaps some folks that have been type=hinting their production code bases
could provide anecdotal evidence.

Anyway, if my hypothesis is correct, then it's not so bad that not-so-nice
syntax is required to type hint general purpose utilities.

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ROPQTGYCTCL6N37MJHFFFWSPPGGS2662/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Christopher Barker]

> On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:
>
> > tl;dr: I find it very troubling that we are going on a path where need
> > to increase the language complexity (syntax) only in the cause
> > 'easier' typing.


Which brings up the question is whether it's worth adding  syntax for
typing, but only in the context of typing. As of right now,
typing.get_type_hints() will evaluate a string annotation, e.g.

In [62]: def f(x:"int"):
...: pass
...:

In [63]: typing.get_type_hints(f)
Out[63]: {'x': int}

so get_type_hints could extend its acceptable syntax with this new use of
-> -- and it could get used by wrapping it in quotes. And depending on
how PEP 563 gets resolved, the quotes may not be necessary in the future.

And this could open up some other nifty things, like extending what's
allowable inside [] -- there was a discussion a while back on python-ideas
about extending the __getitem__ protocol, partly motivated by type hints.

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JWPMHLSPFTYAYSMWCX7LOXWBEU4FA6DC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Batuhan Taskaya]

I think I've posted it as a standalone reply (sorry for my bad mailing list 
skills), so linking my thread about potential concerns/reservations: 
https://mail.python.org/archives/list/python-dev@python.org/thread/FI4AFU3I25PECARIH2EVKAD5C5RJRE2N/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4QFTYRBXNNIIJ7S3ZXDNBDZS63OBHLQT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Paul Moore]

I agree. The same concerns and reservations apply for me.

On Sat, 18 Dec 2021 at 21:13, Terry Reedy  wrote:
>
> Batuhan expresses my concerns better than I could, so I just add my
> agreement.
>
> On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:
>
> > tl;dr: I find it very troubling that we are going on a path where need
> > to increase the language complexity (syntax) only in the cause
> > 'easier' typing. So I am opposed to this change.
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/G6PD5KYVI3CCAPFEOQ3VT7QJAN3Y6Z6W/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LS7DT7QVJ5RO5VX4IVCBMVQILGD2UQMQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Terry Reedy]

Batuhan expresses my concerns better than I could, so I just add my 
agreement.


On 12/18/2021 3:13 PM, Batuhan Taskaya wrote:


tl;dr: I find it very troubling that we are going on a path where need
to increase the language complexity (syntax) only in the cause
'easier' typing. So I am opposed to this change.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/G6PD5KYVI3CCAPFEOQ3VT7QJAN3Y6Z6W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: RFC on Callable Syntax PEP

[Batuhan Taskaya]

First of all, thank you for authoring a really well-written PEP.

tl;dr: I find it very troubling that we are going on a path where need
to increase the language complexity (syntax) only in the cause
'easier' typing. So I am opposed to this change.

Most of the native typing use cases until now (PEP 585 / PEP 604) was
using the existing syntax, with slight adjustments on runtime. And
personally, I really liked them (being able to write `list[int | str]`
is nice), but not as much as I like the simplicity of the language. I
think the new syntax proposal for only a typing use case is really
destructive when we already have a very formal way of creating
callable types. I personally don't think it is verbose (as claimed by the PEP),
since the only thing that we have an extra is the `Callable` name. Aside from
these concerns, a few comments on the actual proposal:

- The syntax itself would probably confuse people, considering in
other languages (like javascript) the arrow functions are a thing that
loosely translates to our lambdas. People might try to write code like
`(x, y) -> x + y`, with the expectation that it would work (since it
is a valid syntax now), but it would immediately throw an error (if
x/y is not defined) and create confusion.

- Callables that return callables look very awkward with a double
arrow. `) -> (int, str) -> R:` (compared to `) -> Callable[[int, str],
R]:`. This was one of the examples that really made me look twice to
understand.

- `async` keyword used to be coupled with `def`/`for`/`with`, but
without any of these, it doesn't feel good to have it as some sort of
prefix to these expressions. I think it would be nice if the PEP can
address how much of the code out there uses `Callable[...,
Awaitable[...]]` syntax, since from what I understand by the current
proposal `async (int, str) -> bool` is the same with `(int, str) ->
Awaitable[bool]`, which is, in theory, a new syntax that we might be
adding for a very restricted use case.

- The parameter part (`(x, y) -> `) seem to be even diverging from an
actual expression (by allowing standalone [double-]starred operations,
and disallowing `(int, ...) ->`), which I think would complicate the
actual parsing process (even though we have an advanced parser in
CPython, that does not mean community maintained projects such as
jedi, parso, black will have one).

- Out of 4 concerns that are listed on the PEP about the current
callable syntax; only 1 is about the actual runtime behaviour, which
even if I did not agree initially, we have other solutions. Like an
open issue on the tracker about making `callable()` generic
https://bugs.python.org/issue42102.

- The other 3 is pretty much the same, claiming the syntax is dense.
Which I partly agree on. We probably could find other solutions (using
tuples to represent the parameter group, etc.), but I do not see those
points as very big deals (at least not as big as adding a very
complicated piece of new syntax).

I would assume some of these were previously discussed during the
typing-sig/python-dev threads, but as someone who recently learned
about this proposal, I feel like this is really overworked for a
problem that is (subjectively, again) small.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FI4AFU3I25PECARIH2EVKAD5C5RJRE2N/
Code of Conduct: http://python.org/psf/codeofconduct/