[Python-Dev] Re: Suggestion: a little language for type definitions
On Sun, Jan 09, 2022 at 08:42:14AM -0800, Christopher Barker wrote: > Perhaps it's worth remembering that this thread spun off one about adding > syntax to Python because the current syntax isn't capable of easily > expressing an important type hinting concept (i.e. Callable). I shall quote the PEP's justification (in part) for the benefit of those who haven't read it. [quote PEP 677] There are a few usability challenges with Callable we can see here: • It is verbose, particularly for more complex function signatures. • It relies on two levels of nested brackets, unlike any other generic type. This can be especially hard to read when some of the type parameters are themselves generic types. • The bracket structure is not visually similar to how function signatures are written. • It requires an explicit import, unlike many of the other most common types like list. Possibly as a result, programmers often fail to write complete Callable types. Such untyped or partially-typed callable types do not check the parameter types or return types of the given callable and thus negate the benefits of static typing. [/quote] These are legitimate issues with the Callable type hint. Does that mean that the existing syntax "isn't capable of easily" expressing callables? I don't think so. "Easily" is subjective, and at the risk of undermining the justification for the PEP, I think it is fairly easy: # type hint for a function of two parameters, the first takes # an int, float or complex, the second a bool, and returns # either a str or bytes. from typing import Callable T = Callable[[int|float|complex, bool], str|bytes] It was harder to write it out in English than to write the type hint. So its not *difficult*, there's one import and one class with a subscript needed. The type system is perfectly capable of expressing that. It is no more hard to read than any other expression of equivalent complexity: from aardvark import Surveying T = Surveying((spam+eggs+cheese, eels), foo*bar) But can we do better? The PEP authors think we can, and I am inclined to agree with them. While it is true that "not everything needs to be a builtin", in the case of typing, we are evolving to use builtins where possible: * we use int, and always have, not typing.Int; * what was once typing.List[int] is now list[int]; * what was once Union[float, int] is now float|int etc, but Callable is the last remaining common compound type without a syntactic shortcut. Its not that declaring callables are *too hard* (let alone impossible) but that we can do better than what we've got. Callable[[int|float|complex, bool], str|bytes] (int|float|complex, bool) -> str|bytes The second looks like executable pseudo-code for a function declaration. It avoids the import and the nested square brackets, and it is shorter without being so terse it becomes cryptic. The improved syntax looks just like a function signature with the parameter names excised: def func(a: int|float|complex, b: bool) -> str|bytes which I think is a win. > So arguing that Python is completely readable for type hints is a bit > off-mark, isn't it? What does "completely readable" mean? If you mean that it is impossible to compose a type-hint of such stupendous complexity and obfuscatory brilliance that even the infamous RFC-822 email address validator regex appears trivial in comparison? http://www.ex-parrot.com/~pdw/Mail-RFC822-Address.html Of course I don't mean that. Type hints are expressions, and expressions can have arbitrary complexity, including "too damn high". I argued that type hints are no harder to read than other expressions of the equivalent complexity. I didn't argue that they can't be abused or that the syntax cannot be improved. More to follow... -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZCOX33PTCQI3TUZPIO4GLLUPZJDXYIE5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Suggestion: a little language for type definitions
On Mon, Jan 10, 2022 at 12:05 PM Steven D'Aprano wrote: > > On Mon, Jan 10, 2022 at 05:39:42AM +1100, Chris Angelico wrote: > > > From my understanding, "x->y" would create a Callable if given two > > *types*, but its meaning if given two other objects is still > > undefined. > > The PEP requires parentheses around the argument list, so that would be > a SyntaxError. That's a small restriction that makes it less clear as an operator, but sure, pretend I wrote "(x)->y" in the examples. > The PEP also states that the arrow syntax would be equivalent to calling > Callable. Callable currently enforces that the return type actually is a > type, but doesn't check the input types. > > (I don't know if that is a deliberate design or an oversight.) > > Assuming that it is an oversight, I would expect that only the following > values would be legal for the x and y objects: > > - a type; > - a string (which gets converted to a ForwardRef); > - None; > > and anything else would result in a TypeError. > TypeError is absolutely fine; it's still part of the core language syntax. >>> 1 @ 3 Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for @: 'int' and 'int' If I want to create my own class that responds to matmul, I can do that, even though core data types will all TypeError; it's part of the core language and has syntactic, if not semantic, meaning in any context. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6I4MFDF7R2NNJFSVZ444OLMWBBNEHFPQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Suggestion: a little language for type definitions
On Mon, Jan 10, 2022 at 05:39:42AM +1100, Chris Angelico wrote: > From my understanding, "x->y" would create a Callable if given two > *types*, but its meaning if given two other objects is still > undefined. The PEP requires parentheses around the argument list, so that would be a SyntaxError. The PEP also states that the arrow syntax would be equivalent to calling Callable. Callable currently enforces that the return type actually is a type, but doesn't check the input types. (I don't know if that is a deliberate design or an oversight.) Assuming that it is an oversight, I would expect that only the following values would be legal for the x and y objects: - a type; - a string (which gets converted to a ForwardRef); - None; and anything else would result in a TypeError. -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OIHDPKBSWSRV3QP2VOKP6PPA7AV65V5P/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] PEP 679 – Allow parentheses in assert statements
Hi everyone, I would like to start a discussion about a small PEP proposal to allow parentheses in assert statements to fix a common gotcha with assert statements. Link to the PEP: https://www.python.org/dev/peps/pep-0679/ *Please, redirect all discussions to: * https://discuss.python.org/t/pep-679-allow-parentheses-in-assert-statements/13003 as I will not be monitoring answers to this thread. Thanks, everyone for your time! Regards from cloudy London, Pablo Galindo Salgado ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/I7MKCD3GHJXCERFCZ2FD3X7IPAX6ASVK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Suggestion: a little language for type definitions
El dom, 9 ene 2022 a las 10:50, Chris Angelico () escribió: > On Mon, Jan 10, 2022 at 3:49 AM Christopher Barker > wrote: > > > If "x->y" is syntactically valid anywhere in Python code, it's not a > > problem that there are no core data types for which it's meaningful.) > > > > Here's where I'm not so sure -- this looks a lot like a binary operator, > but it behaves quite differently. IIUC it would always create a Callable, > regardless of what the types were of the two other types. And it would not > invoke a dinder on either, yes. > > > > Nor would it be like assignment. > > > > This is even worse than the use of [] in type hinting which is also > using the same sytax for a very different meaning -- at least that one is > stil calling __getitem__ :-) > > > > From my understanding, "x->y" would create a Callable if given two > *types*, but its meaning if given two other objects is still > undefined. So there's still room for it to be an operator, just like > [] is, and for it to be given semantic meaning for the 'type' type and > all of its subclasses. Or alternatively, there's room for it to be > given meaning in a completely different way, but still universally > (there's a proposal for it to be a form of inline function, although > I'm not 100% sure of the details there). > With the current iteration of PEP 677, `(x) -> y` would return a special object (defined in the `types` module) that simply holds whatever `x` and `y` evaluate to. You could put whatever expression you want at runtime and it would work, though a static checker may be unhappy with you. > > ChrisA > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/R6OIHWU5226BANUXMJLZID3KNUDBM4BL/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U6C7Z473YEJYGSN7PKYLQQGEQIH2QRN2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Suggestion: a little language for type definitions
On 9 Jan 2022 at 02:22:31, Steven D'Aprano wrote:
> On Sat, Jan 08, 2022 at 12:59:38AM +0100, jack.jan...@cwi.nl wrote:
>
> I posted this suggestion earlier in the callable type syntax discussion,
> at which point it was completely ignored. Possibly because it’s a really
> stupid idea, but let me post it again on the off chance that it isn’t a
> stupid idea but was overlooked.
>
>
> > If I can make a wild suggestion: why not create a little language
>
> > for type specifications?
>
>
> Any time we are tempted to prefix a question with "Why not ...", the
> question is backwards. The right question is, "why should we ...".
>
> Python is 30 years old and mature, with a HUGE ecosystem of users,
> libraries, tooling, etc. It is far, far easier to get changes wrong than
> to get them right, which is why we should be conservative when it comes
> to language changes (including syntax). Changes to the language are
> "default deny", not "default accept", and it is up to the proponents of
> the change to prove their case, not for opponents to disprove it.
>
I agree 100% on these principles.
As a counter-argument, in this specific case, I’d say that the Python
typing ecosystem is only ~5 years old and still evolving rapidly, so these
principles apply with less force.
I personally believe that the syntax for type annotation doesn’t have, from
the point of view of users, to be strictly Python, as long as it’s
pythonic.
I can sympathise with the idea of repurposing the regular Python syntax to
describe types, and agree that it makes easier to bootstrap tools
(typecheckers, IDEs…) but I think it has its limits in terms of
expressivity, first.
Second, I think willing to keep the same syntax constructs for the sake of
lowering the cognitive burden of new developers is a red herring. These are
two different languages, with entirely different semantics, and the links
between them is not complement intuitive.
As an example (just for the sake of argument, I’m not proposing this kind
of change) one could argue that 'list[int]’ doesn’t really relate to how
lists are constructed or how the bracket operator is used in regular
Python. A more intuitive construct (from this point of view) could have
been ‘[int…]’. Same for dictionaries: ‘dict[int, str]’ vs. ‘{int: str}’
which looks closer that how dictionaries are constructed in Python.
S.
--
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Co-Founder & Co-Chairman, National Council for Free & Open Source Software
(CNLL) - http://cnll.fr/
Co-Founder & Chairman, Association Professionnelle Européenne du Logiciel
Libre (APELL) - https://www.apell.info/
Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) -
https://www.euclidia.eu/
Founder, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/V3T7JISJOSXUDBB3XZJLPG4FBZQTBSUY/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Suggestion: a little language for type definitions
On Mon, Jan 10, 2022 at 3:49 AM Christopher Barker wrote: > > If "x->y" is syntactically valid anywhere in Python code, it's not a > problem that there are no core data types for which it's meaningful.) > > Here's where I'm not so sure -- this looks a lot like a binary operator, but > it behaves quite differently. IIUC it would always create a Callable, > regardless of what the types were of the two other types. And it would not > invoke a dinder on either, yes. > > Nor would it be like assignment. > > This is even worse than the use of [] in type hinting which is also using the > same sytax for a very different meaning -- at least that one is stil calling > __getitem__ :-) > >From my understanding, "x->y" would create a Callable if given two *types*, but its meaning if given two other objects is still undefined. So there's still room for it to be an operator, just like [] is, and for it to be given semantic meaning for the 'type' type and all of its subclasses. Or alternatively, there's room for it to be given meaning in a completely different way, but still universally (there's a proposal for it to be a form of inline function, although I'm not 100% sure of the details there). ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/R6OIHWU5226BANUXMJLZID3KNUDBM4BL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] [PSA] OpenSSL 3.0 support is preliminary and experimental
Hi, I would like to remind everybody that Python's support for OpenSSL 3.0 is preliminary [1]. Python compiles with OpenSSL 3.0.0 and simple code kinda works. However there are known performance regressions, missing features (e.g. usedforsecurity flag), and potential bugs cause by API incompatibilities. Due to the experimental state I advise against using Python with OpenSSL 3.0 in production. It may take a while until Python gains full support for the next version of OpenSSL. I have shifted my personal OSS time to more fun topics like performance and WASM. My work time is currently limited, too. Christian [1] https://docs.python.org/3/whatsnew/3.10.html#ssl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ATO4DM6QYZGLSGGDZ3TRN5X3QDD5OHOE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Suggestion: a little language for type definitions
Perhaps it's worth remembering that this thread spun off one about adding syntax to Python because the current syntax isn't capable of easily expressing an important type hinting concept (i.e. Callable). So arguing that Python is completely readable for type hints is a bit off-mark, isn't it? The question at hand is whether changes in syntax that are desirable for type hinting should be applied across the board in all of Python. I think the debate boils down to: Is it more clear for readers to have two different (but related) related syntaxes for two different (but related) purposes, or to have one Sytax that is used in different ways? Chris A addresses this specifically: > To that extent, I definitely want to keep annotation syntax and Python syntax the same; if there's a new feature needed for annotations, add it to the base language, even without any useful semantics. And here's where reasonable people can disagree :-) > (Valid syntax without semantics is what we have with the matmul operator. I can syntactically parse "f@g(y)" because I know how the matmul operator works, even without knowing the data types involved. Sure: because it's a binary operator like all the other binary operators. > If "x->y" is syntactically valid anywhere in Python code, it's not a problem that there are no core data types for which it's meaningful.) Here's where I'm not so sure -- this looks a lot like a binary operator, but it behaves quite differently. IIUC it would always create a Callable, regardless of what the types were of the two other types. And it would not invoke a dinder on either, yes. Nor would it be like assignment. This is even worse than the use of [] in type hinting which is also using the same sytax for a very different meaning -- at least that one is stil calling __getitem__ :-) -CHB On Sat, Jan 8, 2022 at 8:46 PM Steven D'Aprano wrote: > On Sat, Jan 08, 2022 at 06:30:53PM -0800, Ethan Furman wrote: > > On 1/8/22 5:46 PM, Steven D'Aprano wrote: > > > > > [...] if you hate type annotations because they are unreadable, then > you > > > hate Python because Python is unreadable. > > > > Not so. > > Are you disputing that annotations use the same syntax as other Python > expressions? If not, I don't see how you can deny that "type annotations > are unreadable" implies "Python expressions are unreadable", which in > turn implies "Python is unreadable". > > > > A simple list comprehension is (usually) quite readable, while a > > triply-nested list comprehension all on one line is not. > > Indeed. We can abuse any syntax. So do we conclude that comprehensions > are "unreadable" because we can write obfuscated triply-nested list > comprehensions? > > > > Similarly, adding type information in between a variable name and its > value > > is not (for me, and apparently others too) readable. > > I think that "unreadable" or "not readable" is a complaint that gets > overused, often for the most trivial cases, to the point that it loses > all credibility. Especially when it comes from people who are fluent in > C (which may not be you, Ethan). > > http://unixwiz.net/techtips/reading-cdecl.html > > "Easily learned", huh. I think that this is one of the clearest examples > of the curse of knowledge as it applies to programming that one could > hope to find. > > Anyway, let's compare: > > # C > int n = 44; > > # Pascal > var > n: integer; > n := 44; > > # Typescript > var n: number = 44; > > # Java > int n = 44; > > # Python > n: int = 44 > > > There are millions who find the C, Pascal, TypeScript and Java perfectly > readable. I don't find it credible that people are incapable of > reading the last one. > > Aside: such a type hint is redundant, as mypy is perfectly capable of > inferring that n = 44 makes n an int. Style guides should recommend > against such redundancy, and I would certainly flag that in a code > review. A better example of a *useful* type hint would be: > > L: list[str] = [] > > > > > Most horribly of all, cluttering a function header with type information > is > > most unreadable. > > I hear you. Adding redundant or unnecessary type hints to parameters > just for the sake of having type hints is just clutter, especially if > they are never tested by actually running a type checker over the file. > > (Untested code is broken code. If not right now, it soon will be.) > > Fortunately, we have *gradual typing*, and nobody should be forced to > use type hints in their projects if they don't want them. Just as we > don't make linters mandatory, we don't make typing mandatory either. > > I think that, outside of very simple functions, once we make the > decision to annotate a function, we should space them out: > > # Better > def func(spam: list[str], > eggs: float, > cheese: str = 'cheddar', > aardvark: str|bytes = "", > eels: Optional[Tuple[int, str]] = None > ) ->
[Python-Dev] Re: Suggestion: a little language for type definitions
> Here is the type hint for `len`, taken from the stub file in typeshed: > > def len(__obj: Sized) -> int: ... > > Putting the mysterious double underscore naming convention aside, I do > not find it credible that anyone capable of programming Python beyond a > beginner level can find that "unreadable". Not by any definition of > unreadable I can think of. Sure, that's pretty trivial, no question. As would be the similar C declaration. As Glenn Lindermann reminded me of cdecl: https://cdecl.org/ you can see how you can get carried away. It's the "getting carried away" parts of the (sometimes organizationally mandatory) type system in Python that are problematic for me, not the simple sized object input, int output sort of thing. You have people asking questions like these: https://discuss.python.org/t/contravariant-typing-type/12741 https://discuss.python.org/t/how-to-annotate-a-new-dict-class-with-typeddict/12723 I don't know if they are just trying to run with scissors, are way the hell off in the weeds, or if the more esoteric corners of tle typing world are simply going to continue to impose themselves on the rest of us. There will always be people who want to express "declare foo as pointer to function (void) returning pointer to array 3 of int" (from the cdecl.org website). Other people have to read that. Maybe Python will eventually grow a pydecl.org domain and website to serve a similar purpose. :-) > Even if your type system is not Turing complete, it is still going to be > pretty powerful. We're not using Pascal any more :-) And that means that > the types themselves are communicating some fairly complex semantics. > Blaming the syntax for something which is inherently hard is not helpful. I don't think anyone's blaming the syntax. I interpreted Jack's suggestion to mean that we would be able to do better with t-strings encapsulating a little language designed to cleanly describe types. I first encountered Python in the 1993-1994 timeframe (1.0.something). Part of its appeal to me at least (and to many others I think) was that it was the anti-Perl. Perl's obfuscation wasn't in its typing. It was elsewhere (everywhere else?). With a full-fledged type system in place it seems like Python is starting to desert that niche. (Yes, I realize Perl is no longer the big dog it once was.) Skip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PPFPZUUWKEBNRDTDSPNCQRC5XENB6Q62/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About vulnerabilities in Cpython native code
On Sun, Jan 9, 2022 at 7:35 PM Stephen J. Turnbull wrote: > > Chris Angelico writes: > > > Not completely, just very minorly. I'm distinguishing between attacks > > that can be triggered remotely, and those which require the attacker > > to run specific Python code. For example, using ctypes > > OK. AFAICT that was a red herring introduced to the thread solely to > support the claim "Python isn't memory-safe [anyway]" so it's not > reasonable to claim a Python bug is a vulnerability. The original > post didn't depend on ctypes or anything like that; it claimed there > *might* be vulnerabilities in CPython's C code. If so, my claim is > that they would indeed be security-relevant, regardless of what users > with access to Python source might or might not be doing. > That's entirely possible, but I eyeballed a number of the examples cited, and they weren't (you can't use an HTTP request to trigger test code, as far as I know). For any of these to be viable issues, they would have to be triggered somehow, and in many cases, it's far from obvious how you might do that. The problem is that this is a single monster report with a huge number of uninteresting concerns (the "value written to but never read" ones), interspersed with a number of complaints which aren't actually issues (like calling Py_CLEAR with a potentially null pointer, which is perfectly safe). It's hard to know whether there are any real issues, without spending a lot of time weeding through the nonissues. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/A5627EPSYKIM7JVD67GXT42SYFFWZX47/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About vulnerabilities in Cpython native code
Chris Angelico writes: > Not completely, just very minorly. I'm distinguishing between attacks > that can be triggered remotely, and those which require the attacker > to run specific Python code. For example, using ctypes OK. AFAICT that was a red herring introduced to the thread solely to support the claim "Python isn't memory-safe [anyway]" so it's not reasonable to claim a Python bug is a vulnerability. The original post didn't depend on ctypes or anything like that; it claimed there *might* be vulnerabilities in CPython's C code. If so, my claim is that they would indeed be security-relevant, regardless of what users with access to Python source might or might not be doing. Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ESNESNCOM72YSPOPWECSCVKFOFYJ7USP/ Code of Conduct: http://python.org/psf/codeofconduct/
