[Python-ideas] Re: Should Python enforce Type-checking in the future?
Hi, (NB: I'm just a lurker here, posting for almost the first time. I may say silly things.) Le 09/12/2021 à 21:32, deavid a écrit : Hi, I would like to hear the opinion of Python's community on enforcing types in the future for the language. I've been using Python as my main language for everything for around 10 years, until I started moving to Rust 2 years ago; one of the main factors was types. Just before moving to Rust I started to use mypy heavily, which I liked a lot and uncovered tons of potential problems. Now (2 years later), it seems the situation hasn't changed much; I might be wrong, so let me know what improvements you think landed in this area in the last 2-3 years. I don't follow typing closely at all, but I do nevertheless see quite a number of typing PEPs getting accepted. Here are a few recent ones gathered just by reading the index in PEP 0: https://www.python.org/dev/peps/pep-0589/ https://www.python.org/dev/peps/pep-0586/ https://www.python.org/dev/peps/pep-0585/ https://www.python.org/dev/peps/pep-0591/ https://www.python.org/dev/peps/pep-0593/ https://www.python.org/dev/peps/pep-0604/ https://www.python.org/dev/peps/pep-0612/ https://www.python.org/dev/peps/pep-0613/ https://www.python.org/dev/peps/pep-0647/ I feel it's possible this topic might cause a lot of passionate answers, but I just want to hear honest opinions on this. I firmly believe that Python's future would be better if types were enforced by default at "compile time" (whatever this means in Python), with an option/flag to disable this, and integrate MyPy or similar into the interpreter. I'm fully aware that a transition like this would be very hard and long, but I don't think it's impossible. Here's a list of my reasons to think that Python is better if it was typed: 1) On really big codebases and complex projects, it's very easy to lose track of what things do. Types help detecting bugs early. (Ask anyone that has used Rust + Clippy, the amount of errors that are catched is amazing, programs tend to work on the first try) 2) Libraries are currently the top bottleneck for any team to start using MyPy/Pytype. Making types mandatory would ensure all libraries have type support. (If anyone has any other proposal to enforce this, I would like to hear) 3) IDE integration is way simpler and better with types. 4) The interpreter could take further optimizations if it can prove that a function or piece of code is guaranteed to have a limited set of types. This could be used by libraries to have great speed ups that currently are not possible. 5) Static analysis tools could also benefit from types to gain more insight on what the code is trying to do. These are benefits of types in general, but I don't see clearly how having checks built as part of the standard CPython interpreter will increase them. Performance perhaps, but the projects using annotations to eliminate runtime overhead are in experimental stage. Asking a genuine question here: is it actually possible to generate optimized code that supports all of Python? What about things like int subclasses? MyPyC doesn't support them as far as I can see, and I don't see how it could when trying to generate optimized code. It might only optimize int operations when it can prove that all callers don't pass int subclasses, but this means recompiling when a new caller does pass a subclass -- which turns Python into far more of a compiled language. Or am I missing something? On the other hand, with the specializing adaptive that is the result of the Faster CPython project, it might become eventually possible to specialize based on types, and have a cheaper code path when the annotation is int that will still fall back to a slower path when an int subclass instance is passed. Also, you seem to imply that having typing more tied to the core language will bring it more popularity. How will this happen exactly? People don't like it when you break their code. If Python 4 causes trouble for users who don't need/want typing, these users will not suddenly start using typing, but simply stay with Python 3. You'd apparently want to require libraries to be annotated for them to support Python 4, which means pain that the ecosystem simply cannot afford, I think. -- And if the annotations remain optional, then I guess I don't really understand how it's going to push people to add them, especially on large libraries, where this is a significant endeavor. All the best, Jean ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YU2CL5VLW5RPPNNKGFMLDDNRJMSXG5U7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
On 2021-12-10 at 20:20:05 +, deavid wrote: > A lot of you are referring to scripting, learning, and other ways of > using Python that would be badly impacted. I already acknowledged that > these exist (or at least I didn't notice here any other that I wasn't > aware), and that's why I mentioned that there should be a way to avoid > type-enforcing for these scenarios. Maybe by flag, by Python version, > file extension... The method doesn't matter much at this moment. If > it were "easy" to continue writing the old programs without types, > what's the problem in trying to make types more popular? I concede that certain programmers believe that type declarations and static type checking are unilaterally and universally beneficial, and that's why I think that there should be a way to request type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter, because in all my decades of writing software, in a dozen or more languages, getting the types right at coding time has never been an issue. It's very easy to get type checking if you want it, what's the problem in trying to make types more popular? [insert your own perfect-joking-serious-balance emoticon (no emoji, please!) here] ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LQ4CQOKHQXI6QTTYSFFCXPVZR26W3MPI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
On Sat, Dec 11, 2021 at 7:44 AM deavid wrote: > When I said earlier in "compile time", I was just trying to refer to the > parsing step, or pre-parsing step. Or maybe means that "python a.py" => "mypy > a.py && python a.py". > I'm being very ambiguous here on purpose because there are tons of ways to do > this and "how to do it" or the cons of a particular method aren't something > that I wanted to discuss, I just wanted to get a general idea of where the > community stands. > You could make a shell alias that does that, if that's what you want. > A lot of you are referring to scripting, learning, and other ways of using > Python that would be badly impacted. I already acknowledged that these exist > (or at least I didn't notice here any other that I wasn't aware), and that's > why I mentioned that there should be a way to avoid type-enforcing for these > scenarios. Maybe by flag, by Python version, file extension... The method > doesn't matter much at this moment. > If it were "easy" to continue writing the old programs without types, what's > the problem in trying to make types more popular? > Or by simply running Python without first running MyPy. That's a method. > > Another project you may want to look at is MyPyC [1], which is similar > > in concept but takes a different approach; it compiles type-annotated > > Python to CPython C extension code. MyPyC is much more mature and > > something you could easily try out today. > > [1] https://github.com/mypyc/mypyc > > There you have it. I'm amazed that this is already a thing and the discussion > of "enforcing types" didn't happen up to now. > If MyPyC can compile to C and get a 4x performance boost, it means that > Python is leaving a 4x boost on the table. When creating the bytecode (old > *.pyc files) Python could create for some functions a second one that > internally doesn't use Python objects, for those cases where the types > actually match. And the GIL could be released (sometimes) during the > execution of those. > (It might look that the above statement is far too simplistic. But it should > be roughly true with lots of caveats.) > >From the MyPyC docs: """These are our near-term focus areas for improving mypyc: * Improved compatibility with Python""" The 4x speed boost is only if you restrict yourself to the things it can handle. That's fine if a project specifically chooses to do that, but not for general work. And it is definitely NOT a reason to push type checking into the core language. Side question: What if someone wants to use a different type checker than MyPy? Why should that one single checker be blessed with language support? Type annotations are intended to be there for *every* type checker to use. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NCZZZNH34SNOZZ434GFKEMI5OCA4XHQ6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
Well, this has a lot of strong opinions already! I didn't expect less :-) I'm not sure how to answer so many thoughts without making this super-long. I'll try to reply in general. I agree that a compile-time python makes no sense, it would be a different language and we would lose 90% of what Python actually is. This is not what I was suggesting (sorry, I might not have been clear on my first email). I'm talking about something similar to what Typescript is. (mypy even when strict, is closer to what Typescript does than to C++/Rust) But there must be a way to skip typing altogether, both in programs and libraries. (It could also be by file extension... just suggesting more stuff here) When I said earlier in "compile time", I was just trying to refer to the parsing step, or pre-parsing step. Or maybe means that "python a.py" => "mypy a.py && python a.py". I'm being very ambiguous here on purpose because there are tons of ways to do this and "how to do it" or the cons of a particular method aren't something that I wanted to discuss, I just wanted to get a general idea of where the community stands. A lot of you are referring to scripting, learning, and other ways of using Python that would be badly impacted. I already acknowledged that these exist (or at least I didn't notice here any other that I wasn't aware), and that's why I mentioned that there should be a way to avoid type-enforcing for these scenarios. Maybe by flag, by Python version, file extension... The method doesn't matter much at this moment. If it were "easy" to continue writing the old programs without types, what's the problem in trying to make types more popular? And yes, types cannot cover everything. I guess this goes for any language, but especially in Python it is really impossible to type everything. But as Typescript has demonstrated (I think), having a bit of types is better than no types at all. MyPy is there for those that want to use it, of course; but needs to be promoted (this one or different tool) and encouraged. Otherwise most python libraries are just impossible to use with MyPy; and using Python without libraries is quite limiting. >> Exactly. I’ve been advocating for Python for decades — and I think that >> static type checking catches lots of bugs, but they are usually shallow >> bugs — that is most get caught the first time you try to ru the code anyway >> — or certainly if you have decent tests. > >That seems to be close to the opinion of Robert C Martin: This is not true unless your testing covers every branch in every possible state input. This also means that your unit testing in dynamic languages needs to be way more extensive than in static languages; because static checking gives you certain proofs of correctness that you don't need to test. More importantly, it removes the burden of remembering to do those test cases for those tricky scenarios. Overall, your tests will be shorter and easier to maintain. But I would say that even with extensive testing still will not cover all bugs that static type checks can catch: static type checks also can cover future bugs, not only present ones. A type check will prevent someone from calling a function/method in unexpected ways. Because type checks serve to define contracts, and 6 months later you might have forgot that the function assumed certain constraints for the input. > Another project you may want to look at is MyPyC [1], which is similar > in concept but takes a different approach; it compiles type-annotated > Python to CPython C extension code. MyPyC is much more mature and > something you could easily try out today. > [1] https://github.com/mypyc/mypyc There you have it. I'm amazed that this is already a thing and the discussion of "enforcing types" didn't happen up to now. If MyPyC can compile to C and get a 4x performance boost, it means that Python is leaving a 4x boost on the table. When creating the bytecode (old *.pyc files) Python could create for some functions a second one that internally doesn't use Python objects, for those cases where the types actually match. And the GIL could be released (sometimes) during the execution of those. (It might look that the above statement is far too simplistic. But it should be roughly true with lots of caveats.) Anyway, I see that people are generally against what I'm suggesting. But on the other hand I also see that there are projects that are working towards faster python implementations via type-checking. I now noticed that PyPI includes filters for typed packages; I didn't recall this from 3 years ago. I wish they promoted typed ones a bit more, but well, at least we can filter. El vie, 10 dic 2021 a las 17:49, Peter Ludemann () escribió: > Steven D'Aprano wrote: > > > > That seems to be close to the opinion of Robert C Martin: > > http://blog.cleancoder.com/uncle-bob/2016/05/01/TypeWars.html > > He also has some comments on languages like Koitlin and Swift that have > > gone down that path of
[Python-ideas] Re: Should Python enforce Type-checking in the future?
On Thu, 9 Dec 2021 at 17:54, deavid wrote: > Hi, I would like to hear the opinion of Python's community on enforcing > types in the future for the language. > here goes opinion: "no"! strong no. while _using_ tooling that have correct typing is a bit easier, it can make it hard to write code, code harder to read, and in some instances, simply make it almost impossible type correctly what once has been one of the best features of the language: things that are broad into what they accept, strict in what they return, while not being ambiguous. As an example, I will let you with the buitin functions "min" and "max". Write teh pure-Python correct annotation for those, and you will understand what people mean by "typing can be painful". Moreover, you missed that there are very different profiles of Python users. Professional coders working in big projects can take advantage of all the points you listed - but a lot of people just want a way to "give orders to the computer". ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/X7WS7IOT5Q7ECXIX3OWXQSEHY5A7PMFS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
Steven D'Aprano wrote:
> > That seems to be close to the opinion of Robert C Martin:
> http://blog.cleancoder.com/uncle-bob/2016/05/01/TypeWars.html
> He also has some comments on languages like Koitlin and Swift that have
> gone down that path of mandatory static typing:
> http://blog.cleancoder.com/uncle-bob/2017/01/11/TheDarkPath.html
Yet another example of "Uncle Bob" writing stuff that's "not even wrong". Of
course, typing doesn't catch all bugs; but neither does Uncle Bob's
testing-testing-testing. And never will. Programmers need all the help they can
in catching bugs and also in understanding other people's code - and types are
really helpful for that ... I can't count the number of hours I've spent at
Google, figuring out how something works so that I can add some functionality -
often by adding stuff like logging.info("Type of qqsv: %s", qqsv.__class_).
(On Mondays, Wednesdays, and Fridays, I think that mandatory typing would be
fabulous (to paraphrase Brian Reid: "most people won't use types even when you
threaten them"). On Tuesdays, Thursdays, and Saturdays, I meditate on how
difficult it is to write types in Haskell, which is far easier for typing than
Python. On Sundays, I go for a head-clearing walk in the park.)
PS: For another example of "Uncle Bob" nonsense:
https://groups.google.com/g/software-design-book/c/Kb5K3YcjIXw/m/qN8txMeOCAAJ
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/L6FPYNFJKFJPD4Z4NHVP2E5BXERDDPT4/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
On Fri, Dec 10, 2021 at 08:32:27AM -0800, Christopher Barker wrote: > On Fri, Dec 10, 2021 at 4:07 AM Steven D'Aprano wrote: > > > > 1) On really big codebases and complex projects, it's very easy to lose > > > track of what things do. Types help detecting bugs early. > > > > Static types help detecting *some* kinds of bugs early. > > > Exactly. I’ve been advocating for Python for decades — and I think that > static type checking catches lots of bugs, but they are usually shallow > bugs — that is most get caught the first time you try to ru the code anyway > — or certainly if you have decent tests. That seems to be close to the opinion of Robert C Martin: http://blog.cleancoder.com/uncle-bob/2016/05/01/TypeWars.html He also has some comments on languages like Koitlin and Swift that have gone down that path of mandatory static typing: http://blog.cleancoder.com/uncle-bob/2017/01/11/TheDarkPath.html -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/374OAS5KUHWI3QS5J4KMXY6XERA45B3D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
On Fri, Dec 10, 2021 at 4:10 AM Carl Meyer wrote: > Another project you may want to look at is MyPyC [1], Or Cython, which has used a non-standard typing system for years, and is currently working on adopting the new types. It’s a very tricky business though, because Cython is generating C code, so it can’t use even semi-dynamic types. E.g. needs to know if something is an actual list, rather than a MutableSequence. I wonder how MyPyC handles that. Honestly, I’m concerned that the Trent towards static typing will turn Python into a very different language, as it’s much easy to use “locked down” static types than truly dynamic ones. E.g list rather than MutableSequence, or even better, the Protocol types. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EIPYUMCJQ3SGNSGL54OMROP74RHCJLK3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
Another project you may want to look at is MyPyC [1], which is similar in concept but takes a different approach; it compiles type-annotated Python to CPython C extension code. MyPyC is much more mature and something you could easily try out today. Carl [1] https://github.com/mypyc/mypyc ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/E66XKIJSQ2XWD4VVLE3CXZ5AJTKFBUVF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
Hi David, I think there's an important distinction to be made between "enforcing types" as in "type annotations that are present are checked for consistency at compile time and enforced at runtime, instead of only being checked by a linter and possibly totally wrong at runtime" vs "enforcing types" as in "everything must always be fully typed." It seems that you are proposing both together, and so far readers are mostly reacting against the second. But I would like to observe that the one does not have to imply the other. I think requiring types everywhere is pretty much a non-starter for the Python language and community; the resulting language would no longer be Python, it would be a very different language with Python's syntax. But I think integrating optional types more closely into the language, so that users can trust that the type annotations that are present are not well-intentioned lies, and the compiler can make use of them to optimize some operations, increases the value of type annotations and the experience of using them, and can still fully allow for untyped and partially-typed code in the places where static types don't make sense (because the code is highly dynamic or because its just a quick script or prototype.) I currently work on a project we call "Static Python" (not a great name, since really it's fully dynamic Python with optional static types) which does exactly this. Types remain fully optional, the type system is gradual, but where you do specify annotations of types that are built-ins or are defined in modules opted into the Static Python compiler, you get compile-time type checking and runtime checks that the values used match the annotated types. This is currently only available if you use the fork of CPython in which we develop it [1]. We hope that at some point it may be possible to instead make it available as a pip-installable extension to normal CPython, but this is probably a long ways out yet. We are building this primarily for the performance benefits. The benefit of being able to use statically known and trusted type information to elide some costly dynamic operations more than makes up for the cost of the added runtime type checks, partly because our compiler is able to intelligently elide those runtime type checks in static-to-static calls, only checking when values first are passed in from untyped sources. This is early experimental work, and very far from anything that might be considered for mainstream CPython, but if you're interested in this area, you might find it worth taking a look at this early stage! Carl [1] https://github.com/facebookincubator/cinder ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZM22AKAFIQDHF2E7VPHQ2DZJNBXPTRFM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
Some additional comments.
On Thu, Dec 09, 2021 at 08:32:43PM +, deavid wrote:
> 1) On really big codebases and complex projects, it's very easy to lose
> track of what things do. Types help detecting bugs early.
Static types help detecting *some* kinds of bugs early.
Python doesn't stop you from running static type checks on your code.
That's what mypy, and other type checkers, already do. What does
integration gain us apart from forcing people who care nothing about
static type checking to care, whether they want to or not?
> (Ask anyone that
> has used Rust + Clippy, the amount of errors that are catched is amazing,
> programs tend to work on the first try)
Isn't that a contradiction? If the work on the first try, then obviously
there weren't any errors to be caught. If there are errors that are
caught and need to be fixed, then it wasn't your first try.
In any case, I can often say the same about Python code, with no static
checking. Typically, after writing some functions, and fixing a handful
of typos -- I've been writing Python more than 20 years and I still
sometimes write {key=value} in dicts -- my unit tests Just Work.
(Unless they don't. *cough*)
But when they don't work, it's almost never *type* checks that fail. Its
logic errors and programming errors in the algorithm, and static typing
doesn't solve those.
> 2) Libraries are currently the top bottleneck for any team to start using
> MyPy/Pytype. Making types mandatory would ensure all libraries have type
> support.
Making types mandatory would more like ensure that many libraries would
just stop being supported and they would move to other languages.
> Of course, types have their own set of drawbacks; for example it could make
> Python look harder to code for newcomers, or it might get in the way for
> things like Jupyter notebooks, ML, and similar stuff.
Right.
> Because of this, an
> escape hatch must always exist. (maybe there are even more problems I am
> not aware about, I'd love to hear)
But if there is an escape hatch, then the types are not mandatory, and
many libraries will not use them, many projects will not use them, and
you have the status quo except mypy is bundled with CPython.
So all you will actually gain is avoiding running
pip install mypy
or whatever command you use to install it. That's a lot of disruption to
the entire Python ecosystem just so people can avoid installing mypy.
--
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/GRI6EDFFWVK355QVU5TDC3PNQEP5D3C4/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
First off I love types. They, as you point out, are a great tool for finding bugs. However, there are reasons why requiring types would either require the use of meaningless types e.g. marking everything as the Any type or require python to solve the halting problem. Consider the following code example: ``` class ArrayMeta(type): On Thu, Dec 9, 2021 at 12:52 PM deavid wrote: > Hi, I would like to hear the opinion of Python's community on enforcing > types in the future for the language. I've been using Python as my main > language for everything for around 10 years, until I started moving to Rust > 2 years ago; one of the main factors was types. > > Just before moving to Rust I started to use mypy heavily, which I liked a > lot and uncovered tons of potential problems. Now (2 years later), it seems > the situation hasn't changed much; I might be wrong, so let me know what > improvements you think landed in this area in the last 2-3 years. > > I feel it's possible this topic might cause a lot of passionate answers, > but I just want to hear honest opinions on this. > > I firmly believe that Python's future would be better if types were > enforced by default at "compile time" (whatever this means in Python), with > an option/flag to disable this, and integrate MyPy or similar into the > interpreter. I'm fully aware that a transition like this would be very hard > and long, but I don't think it's impossible. > > Here's a list of my reasons to think that Python is better if it was typed: > > 1) On really big codebases and complex projects, it's very easy to lose > track of what things do. Types help detecting bugs early. (Ask anyone that > has used Rust + Clippy, the amount of errors that are catched is amazing, > programs tend to work on the first try) > 2) Libraries are currently the top bottleneck for any team to start using > MyPy/Pytype. Making types mandatory would ensure all libraries have type > support. (If anyone has any other proposal to enforce this, I would like to > hear) > 3) IDE integration is way simpler and better with types. > 4) The interpreter could take further optimizations if it can prove that a > function or piece of code is guaranteed to have a limited set of types. > This could be used by libraries to have great speed ups that currently are > not possible. > 5) Static analysis tools could also benefit from types to gain more > insight on what the code is trying to do. > > Of course, types have their own set of drawbacks; for example it could > make Python look harder to code for newcomers, or it might get in the way > for things like Jupyter notebooks, ML, and similar stuff. Because of this, > an escape hatch must always exist. (maybe there are even more problems I am > not aware about, I'd love to hear) > > If it were for me, I would like to have a Python 4 that is exactly a > Python 3 but with mypy bundled and strictly enforced by default; with a > flag to convert errors into warnings or disable entirely. Then every > release, say a Py3.11, would also get a Py4.11-beta (the beta would be to > avoid people migrating until it's ready). > > In this way, for a library to say it has Py4 compatibility it would need > to be type-ready. Jupyter notebooks and such would be stuck at Py3, but of > course, getting all the releases; and enterprises would be trying to use > Py4 whenever it were ready. > > Typescript is also basically Javascript with types (well, not only that, > but anyway) and the result is quite good. In this fashion, another > alternative is having a second binary called TPython or MPython, and > include it on the regular Python distribution; this would cause less push > to go for types, but it could do the trick too. > > So well, my question here is: why is this not a thing? Has anyone proposed > something like this before? I feel I must have missed something important. > > Thanks, > David > > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/QBWYBXYK7NHSLQYWPV6PXAGIR4JB4FWU/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7IW6OMFPNZWHBGL4Z22VFZPCB7FDVK7L/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
"Should Python enforce Type-checking in the future?" Python already enforces type-checking. It is *runtime* time checking. https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ As far as *static* type checking goes, the answer is No. (Obviously all of the following is my opinion, not holy writ.) (1) Moving mypy into CPython would be bad for both mypy and CPython. The release schedules and stability guarantees of each are different. It would require the influx of a huge amount of complexity into CPython to integrate mypy. And to save what? mypy is free, and easy to install if you have an internet connection. The benefits of integration are tiny compared to the costs. (2) Requiring static type checking would be bad for alternative implementations. Python is dominated by CPython, and we should be mildly concerned about that: languages dominated by a single implementation (like Ruby and Perl) are far more likely to lose popularity and fade away than those with many implementations (like C). Anything that makes it harder for new implementations to get started is a negative. Right now, we do have PyPy and MicroPython, both reasonably popular, but the other "big" implementations Jython and IronPython are stuck in the Python 2 world. Making it harder for them to catch up would be a negative. How hard would it be to migrate PyPy to enforced static typing? What would that do to the smaller Python implementations? How would it affect Cython and Numba and other projects? MicroPython needs to run on tiny machines with limited memory resources. It will probably never support mandatory static type checking. Running Python on tiny machines like Raspberry Pi and Arduino is a big part of the Python ecosystem, so mandatory static checking would break the language and ecosystem into what is effectively two languages: classic Python 3 and new "Python 4" with mandatory static checking. (3) Have we learned nothing from the 2-to-3 transition of the pain and extra work and disruption that a huge backward-compatibility breaking change to the language makes? Faced with a major migration from 3-to-4 that would be as big and painful as 2-to-3, many projects and libraries would just abandon Python altogether, in favour or rewriting their project in a "real" statically typed language with proper stability guarantees. We should not be talking about Python 4 here, we should be thinking Python 5000 or even Python 6000. More seriously, incremental changes to the language is fine. Massive breakage all at once, let's not repeat that. (4) Mandatory static types would be bad for students and beginners, bad for scientists, bad for casual users, and bad for system administrators who still use Python as a scripting language. It would make the language harder to use and harder to learn. Python is, not unique, but very unusual in the broad range of uses people put it to. It is a *teaching language* that can be used for professional projects. It is a *glue language* and a *scripting language* that is written by people who just want something that works and don't want to fight the compiler's type checking errors to run it. That's why dynamic type checking was invented in the first place. (5) Requiring static type checking would be bad for mypy. mypy is designed for *gradual typing*, that is, type checking when only parts of your code have static type hints. To move to a world where type hints are no longer hints but mandatory strictly enforced types would likely make a lot of mypy obsolete. It would likely take a lot of work to rip out the code that assumes gradual typing and replace it with code that strictly enforces types. (6) Requiring formal static types goes against modern language design for applications languages. Python is not a systems language like Rust. Modern applications languages tend to converge to the middle of the typing landscape: - languages with static types tend to develop ways to escape the type system and allow some dynamic typing features; e.g. Java has generics and reflection. - languages with dynamic typing tend to develop optional ways to enforce those types statically. This is, of course, exactly where Python is now. A fundamentally dynamically typed language with optional static typing features. And that's a good thing. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5GJVGEWWUB6WHY2JGQ547UIIWDZQHYCE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
On Thu, 9 Dec 2021 at 20:55, deavid wrote: > > So well, my question here is: why is this not a thing? Has anyone proposed > something like this before? I feel I must have missed something important. IMO, making types mandatory would severely penalise the use of Python for scripting, exploratory coding, and similar adhoc work. This is a significant part of the user base, and we have to consider such users. Having typing as optional allows those people who get benefits from typing to achieve those benefits without impacting others who don't need that level of strictness. So basically -1 from me. Paul ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RKKBRWC5WTHUQNPAF3QUWISTXZMISHHP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should Python enforce Type-checking in the future?
Oops sorry accidentally sent mid typing:
```
class Array:
def __class_getitem__(cls, size):
# I know this usage is discouraged
if hasattr(cls, 'size'):
raise TypeError('cannot resive type')
return type(cls)(f'Array[{size}]', (), {'size': size})
def __new__(cls, *args, **kwargs):
if not hasattr(cls, 'size'):
raise TypeError('cannot instance unsized array')
return super().__new__(cls, *args, **kwargs)
# The rest of the implementation is left
# as an execersize for the reader
def concat(a: Array, b: Array) -> Array:
return Array[a.size + b.size](...)
def make_zeros(i: int) -> Array:
return Array[i](0 for _ in range(i))
# Both of these functions are typed incorrectly.
# Array[i] is not a subtype of Array
# Granted we could change the construction of Array[i]
# to use bases=(cls,) instead of bases=()
```
It also possible to write a function that costructs an entirely new class:
```
def make_new() -> Any :
class A: pass
return A()
```
there is no possible to way to type it more specifically than returning
Any (or object).
Now if libraries are going to be forced to just throw Any everywhere what's
the point?
-- Caleb Donovick
On Thu, Dec 9, 2021 at 3:58 PM Caleb Donovick
wrote:
> First off I love types. They, as you point out, are a great tool for
> finding bugs. However, there are reasons why requiring types would either
> require the use of meaningless types e.g. marking everything as the Any
> type or require python to solve the halting problem.
>
> Consider the following code example:
>
> ```
> class ArrayMeta(type):
>
>
>
> On Thu, Dec 9, 2021 at 12:52 PM deavid wrote:
>
>> Hi, I would like to hear the opinion of Python's community on enforcing
>> types in the future for the language. I've been using Python as my main
>> language for everything for around 10 years, until I started moving to Rust
>> 2 years ago; one of the main factors was types.
>>
>> Just before moving to Rust I started to use mypy heavily, which I liked a
>> lot and uncovered tons of potential problems. Now (2 years later), it seems
>> the situation hasn't changed much; I might be wrong, so let me know what
>> improvements you think landed in this area in the last 2-3 years.
>>
>> I feel it's possible this topic might cause a lot of passionate answers,
>> but I just want to hear honest opinions on this.
>>
>> I firmly believe that Python's future would be better if types were
>> enforced by default at "compile time" (whatever this means in Python), with
>> an option/flag to disable this, and integrate MyPy or similar into the
>> interpreter. I'm fully aware that a transition like this would be very hard
>> and long, but I don't think it's impossible.
>>
>> Here's a list of my reasons to think that Python is better if it was
>> typed:
>>
>> 1) On really big codebases and complex projects, it's very easy to lose
>> track of what things do. Types help detecting bugs early. (Ask anyone that
>> has used Rust + Clippy, the amount of errors that are catched is amazing,
>> programs tend to work on the first try)
>> 2) Libraries are currently the top bottleneck for any team to start using
>> MyPy/Pytype. Making types mandatory would ensure all libraries have type
>> support. (If anyone has any other proposal to enforce this, I would like to
>> hear)
>> 3) IDE integration is way simpler and better with types.
>> 4) The interpreter could take further optimizations if it can prove that
>> a function or piece of code is guaranteed to have a limited set of types.
>> This could be used by libraries to have great speed ups that currently are
>> not possible.
>> 5) Static analysis tools could also benefit from types to gain more
>> insight on what the code is trying to do.
>>
>> Of course, types have their own set of drawbacks; for example it could
>> make Python look harder to code for newcomers, or it might get in the way
>> for things like Jupyter notebooks, ML, and similar stuff. Because of this,
>> an escape hatch must always exist. (maybe there are even more problems I am
>> not aware about, I'd love to hear)
>>
>> If it were for me, I would like to have a Python 4 that is exactly a
>> Python 3 but with mypy bundled and strictly enforced by default; with a
>> flag to convert errors into warnings or disable entirely. Then every
>> release, say a Py3.11, would also get a Py4.11-beta (the beta would be to
>> avoid people migrating until it's ready).
>>
>> In this way, for a library to say it has Py4 compatibility it would need
>> to be type-ready. Jupyter notebooks and such would be stuck at Py3, but of
>> course, getting all the releases; and enterprises would be trying to use
>> Py4 whenever it were ready.
>>
>> Typescript is also basically Javascript with types (well, not only that,
>> but anyway) and the result is quite good. In this fashion, another
>> alternative is having a second binary called TPython or MPython, and
>> include it on
[Python-ideas] Re: Should Python enforce Type-checking in the future?
Please, no, No, NO! It has always been the policy that typing is, and will remain, optional. On 09/12/2021 20:32, deavid wrote: Hi, I would like to hear the opinion of Python's community on enforcing types in the future for the language. I've been using Python as my main language for everything for around 10 years, until I started moving to Rust 2 years ago; one of the main factors was types. Just before moving to Rust I started to use mypy heavily, which I liked a lot and uncovered tons of potential problems. Now (2 years later), it seems the situation hasn't changed much; I might be wrong, so let me know what improvements you think landed in this area in the last 2-3 years. I feel it's possible this topic might cause a lot of passionate answers, but I just want to hear honest opinions on this. I firmly believe that Python's future would be better if types were enforced by default at "compile time" This can't be done, in view of Python's dynamic nature. As I understand it, it is impossible to check at compile time the type of an argument that is about to be passed to a function, unless it is a constant (e.g. a literal int, float, str etc.) Best wishes Rob Cliffe (whatever this means in Python), with an option/flag to disable this, and integrate MyPy or similar into the interpreter. I'm fully aware that a transition like this would be very hard and long, but I don't think it's impossible. Here's a list of my reasons to think that Python is better if it was typed: 1) On really big codebases and complex projects, it's very easy to lose track of what things do. Types help detecting bugs early. (Ask anyone that has used Rust + Clippy, the amount of errors that are catched is amazing, programs tend to work on the first try) 2) Libraries are currently the top bottleneck for any team to start using MyPy/Pytype. Making types mandatory would ensure all libraries have type support. (If anyone has any other proposal to enforce this, I would like to hear) 3) IDE integration is way simpler and better with types. 4) The interpreter could take further optimizations if it can prove that a function or piece of code is guaranteed to have a limited set of types. This could be used by libraries to have great speed ups that currently are not possible. 5) Static analysis tools could also benefit from types to gain more insight on what the code is trying to do. Of course, types have their own set of drawbacks; for example it could make Python look harder to code for newcomers, or it might get in the way for things like Jupyter notebooks, ML, and similar stuff. Because of this, an escape hatch must always exist. (maybe there are even more problems I am not aware about, I'd love to hear) If it were for me, I would like to have a Python 4 that is exactly a Python 3 but with mypy bundled and strictly enforced by default; with a flag to convert errors into warnings or disable entirely. Then every release, say a Py3.11, would also get a Py4.11-beta (the beta would be to avoid people migrating until it's ready). In this way, for a library to say it has Py4 compatibility it would need to be type-ready. Jupyter notebooks and such would be stuck at Py3, but of course, getting all the releases; and enterprises would be trying to use Py4 whenever it were ready. Typescript is also basically Javascript with types (well, not only that, but anyway) and the result is quite good. In this fashion, another alternative is having a second binary called TPython or MPython, and include it on the regular Python distribution; this would cause less push to go for types, but it could do the trick too. So well, my question here is: why is this not a thing? Has anyone proposed something like this before? I feel I must have missed something important. Thanks, David ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QBWYBXYK7NHSLQYWPV6PXAGIR4JB4FWU/ Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/C73JYWGDXPWKHHMFQZ3OQNGKKJFYBUSA/ Code of Conduct: http://python.org/psf/codeofconduct/
