[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Irit Katriel added the comment: Created issue45544 to close all open issues and list them there. -- nosy: +iritkatriel ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Łukasz Langa added the comment: New changeset fdbdf3f7359832820a11ece4c4b01581004d6fe7 by Gregory P. Smith in branch 'main': bpo-40360: Make the 2to3 deprecation more obvious. (GH-29064) https://github.com/python/cpython/commit/fdbdf3f7359832820a11ece4c4b01581004d6fe7 --

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Gregory P. Smith : -- pull_requests: +27331 pull_request: https://github.com/python/cpython/pull/29064 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Łukasz Langa added the comment: The "pending" deprecation status of lib2to3 in 3.9 and 3.10 is no worse than a vanilla deprecation in terms of visibility. It will appear just the same when run with pytest or `-X dev`. However, upgrading the deprecation between 3.10.0rc1 and 3.10.0rc2 really

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

STINNER Victor added the comment: I close the issue: lib2to3 is now deprecated in Python 3.11. I propose to open a new issue in Python 3.13 or newer to remove it. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

miss-islington added the comment: New changeset 559af7434668e2950c08389515a52eba697ef6af by Miss Islington (bot) in branch '3.10': bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122) https://github.com/python/cpython/commit/559af7434668e2950c08389515a52eba697ef6af

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by miss-islington : -- pull_requests: +26566 pull_request: https://github.com/python/cpython/pull/28127 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Łukasz Langa added the comment: New changeset f0b63d5b56a6324f5f86807d9548c7b38aa2a8f7 by Łukasz Langa in branch 'main': bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122) https://github.com/python/cpython/commit/f0b63d5b56a6324f5f86807d9548c7b38aa2a8f7 --

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Łukasz Langa added the comment: > We can add to the 3.10 docs that it is deprecated without any code change. This was already the case: https://docs.python.org/3/library/2to3.html#module-lib2to3 The wording was a bit clumsy so I rephrased in GH-28122. --

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Łukasz Langa : -- pull_requests: +26563 pull_request: https://github.com/python/cpython/pull/28122 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Łukasz Langa added the comment: New changeset d589a7e7eb56196c05337d37417479375878b127 by Victor Stinner in branch 'main': bpo-40360: Deprecate the lib2to3 package (GH-28116) https://github.com/python/cpython/commit/d589a7e7eb56196c05337d37417479375878b127 -- nosy: +lukasz.langa

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: We can add to the 3.10 docs that it is deprecated without any code change. And in 3.11 we can add a warning. -- ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

STINNER Victor added the comment: Guido: How come the deprecation didn't happen in 3.10? Were people just not interested? Well, if nobody deprecates it, it's not deprecated. It is simple as it it :-) IMO it's ok to only deprecate it in Python 3.11, unless Pablo *really* wants to deprecate

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Gregory P. Smith added the comment: I think we just forgot to make the change in time. 3.11 is fine. We're not _maintaining_ lib2to3 or describing it as fit for any modern purpose regardless. It's just code that'll sit around in the back of the 3.10 stdlib but not be able to parse the

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: How come the deprecation didn't happen in 3.10? Were people just not interested? -- ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

STINNER Victor added the comment: I retarget this issue to Python 3.11, since lib2to3 is *not* deprecated in Python 3.10. -- versions: +Python 3.11 -Python 3.10 ___ Python tracker

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

STINNER Victor added the comment: I created PR 28116 to deprecate the lib2to3 package: replace PendingDeprecationWarning to DeprecationWarning. In 2021, I don't think that we should keep the 2to3 tool in the stdlib. Python 2 reached end of line 1 year ago. For the lib2to3 *parser*, IMO it

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by STINNER Victor : -- nosy: +vstinner nosy_count: 11.0 -> 12.0 pull_requests: +26557 stage: -> patch review pull_request: https://github.com/python/cpython/pull/28116 ___ Python tracker

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Gregory P. Smith added the comment: status: lib2to3 PendingDeprecationWarning shipped in 3.9. Since we don't have a specific release planned for the final deprecation, I'll leave this issue open while we figure that out. Once we do, we should promote this to a regular DeprecationWarning

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Gregory P. Smith : -- assignee: gregory.p.smith -> stage: patch review -> ___ Python tracker ___ ___ Python-bugs-list

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: New changeset fe928b32daca184e16ccc0ebdc20314cfa776b98 by Karthikeyan Singaravelan in branch '3.9': [3.9] bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. (GH-21694) (GH-21697)

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Karthikeyan Singaravelan : -- pull_requests: +20841 pull_request: https://github.com/python/cpython/pull/21697 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Karthikeyan Singaravelan added the comment: New changeset cadda52d974937069eeebea1cca4229e2bd400df by Karthikeyan Singaravelan in branch 'master': bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. (GH-21694)

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by miss-islington : -- nosy: +miss-islington nosy_count: 10.0 -> 11.0 pull_requests: +20840 pull_request: https://github.com/python/cpython/pull/21696 ___ Python tracker

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Karthikeyan Singaravelan : -- pull_requests: +20838 pull_request: https://github.com/python/cpython/pull/21694 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

David Halter added the comment: @gvanrossum > Does parso have to be pure Python? If not, we could generate C code like we > do for CPython's parser. I would rather write the parser either in C or Rust. So no, parso does not need to be pure Python. > Now, that doesn't work for incremental

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: Okay, so if you know what to do please do it. ;-) -- ___ Python tracker ___ ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Karthikeyan Singaravelan added the comment: I was referring to PR https://github.com/python/cpython/pull/19663 (commit-503de7149d03bdcc671dcbbb5b64f761bb192b4d) that was merged as part of this issue. It started emitting PendingDeprecationWarning but was not silenced in the test. --

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: Which patch are you referring to? Is it already merged? -- ___ Python tracker ___ ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Karthikeyan Singaravelan added the comment: After this patch test_lib2to3 generates a PendingDeprecationWarning. It can be silenced as it's intentional to avoid test failures while running tests with -Werror. ./python.exe -Wall -m test test_lib2to3 0:00:00 load avg: 2.31 Run tests

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: I guess the design space is wide open. Does parso have to be pure Python? If not, we could generate C code like we do for CPython's parser. Now, that doesn't work for incremental parsing, but I did an alternative implementation that uses a stack machine,

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

David Halter added the comment: Parso's incremental parser is a terrible idea. It also works and is pretty fast, but the design is pretty terrible (it took me a lot of fuzzing to make sure that it works decently well). The basic problem is that it's reusing nodes in a mutable way. If I were

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by wyz23x2 : -- versions: +Python 3.10 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: Thanks for joining in! How do you do incremental parsing with LL1 currently? FWIW I found https://ohmlang.github.io/pubs/sle2017/incremental-packrat-parsing.pdf which may have some useful ideas. -- ___ Python

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

David Halter added the comment: I'm the maintainer of parso. Feel free to addd me to the Nosy List if we have these discussions in the future. Parso is indeed a lib2to3 fork with error recovery, round tripping and incremental parsing as its features. Most pgen2 code has been rewritten since

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Peter Ludemann added the comment: Yes, I'm thinking of doing this as a wrapper, in such a way that it could be incorporated into Lib/ast.py eventually. (Also, any lib2to3-ish capabilities would probably not be suitable for inclusion in the stdlib, at least not initially ... but I have no

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: Can that be done as a 3rd party wrapper? Then you would be able to support older Python versions, and typed_ast (which can parse older Python grammars with a newer Python that's older than 3.8). Plus it would be much easier to get your code released -- no

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Peter Ludemann added the comment: I've written up a proposal for adding "whitespace" handling to the ast module: https://mail.python.org/archives/list/python-id...@python.org/thread/X2HJ6I6XLIGRZDB27HRHIVQC3RXNZAY4/ I don't think it's a "summer-of-code-sized project", mainly because I already

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: There's no python-dev discussion; if you want more feedback I recommend starting on python-ideas first (on either forum you may expect pushback because this is not about a proposed change to Python or its workflow). The Lib/ast.py module will continue to

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Peter Ludemann added the comment: Looking at the suggested successor tools (redbaron, libCST, parso, awpa) ... all of them appear to use some variant of pgen2. But at some point Python will be using a PEG approach (PEP 617), and therefor the pgen2 approach apparently won't work. For a

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

STINNER Victor added the comment: FYI the autopep8 project uses lib2to3. -- nosy: +vstinner ___ Python tracker ___ ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Gregory P. Smith added the comment: New changeset 18f1c60a1625d341a905c7e07367c32c08f222df by Miro Hrončok in branch 'master': bpo-40360: Add a What's New entry for lib2to3 pending deprecation (GH-19898) https://github.com/python/cpython/commit/18f1c60a1625d341a905c7e07367c32c08f222df

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Miro Hrončok : -- pull_requests: +19209 pull_request: https://github.com/python/cpython/pull/19898 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Miro Hrončok added the comment: Thanks for the explanation. I plan to send a PR to add this to the What's new in 3.9 page early next week. Anyone, feel free to beat me to it. -- ___ Python tracker

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: IIRC PendingDeprecationError does not mean that the decision hasn't been made yet. It just means it's less urgent for folks to worry about. I believe we tend to change PendingDeprecationError to DeprecationError in the last release before something is

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Miro Hrončok added the comment: > Getting rid of PendingDeprecationWarning seems like an orthogonal decision; > if it happens, this can trivially be upgraded to DeprecationWarning as part > of a removal sweep. My thought was that the decision was already made to do so. Hence adding new

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: A "What's New" entry would go into Doc/whatsnew/3.9.rst and is much more visible to users looking for exciting bits in the new release (the NEWS file is very large, see e.g. https://docs.python.org/3/whatsnew/changelog.html#changelog. The What's New doc

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Carl Meyer added the comment: > Coul you please add a what's new entry for this change? The committed change already included an entry in NEWS. Is a "What's New" entry something different? > I don't understand why there is a PendingDeprecationWarning and not a > DeprecationWarning. Purely

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Miro Hrončok added the comment: I don't understand why there is a PendingDeprecationWarning and not a DeprecationWarning. See https://discuss.python.org/t/pendingdeprecationwarning-is-really-useful/1038/4 and issue36404 -- ___ Python tracker

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Miro Hrončok added the comment: Coul you please add a what's new entry for this change? -- nosy: +hroncok ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Carl Meyer added the comment: Right, although I think it still makes sense to link both LibCST and parso since they provide different levels of abstraction that would be suitable for different types of tools (e.g. I would rather write an auto-formatter on top of parso, because LibCST's

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: It's typically not up to the core devs to pick a winning third party library; we tend to recommend libraries that are already essentially category winners, like requests. In a sense pointing to LibCST *and* parso is redundant because LibCST builds on

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Peter Ludemann added the comment: The documentation change gives two possible successors: https://libcst.readthedocs.io/ (https://github.com/Instagram/LibCST) https://parso.readthedocs.io/ And I've also seen this mentioned: https://github.com/pyga/awpa Is it possible to settle on one of

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Peter Ludemann : -- nosy: +Peter Ludemann ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Dong-hee Na : -- pull_requests: -19032 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Dong-hee Na : -- nosy: +corona10 nosy_count: 5.0 -> 6.0 pull_requests: +19032 pull_request: https://github.com/python/cpython/pull/18245 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Gregory P. Smith added the comment: I think what we're doing with the documentation update is fine. We can add a warning on stderr to the tool in 3.11. But I don't expect people will be using the tool _from_ the latest CPython 3.x by then. 2to3 is already included with Python 2.7 and the

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Carl Meyer added the comment: @gregory.p.smith What do you think about the question I raised above about how to make this deprecation visible to users of the 2to3 CLI tool, assuming the plan is to remove both? -- ___ Python tracker

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Gregory P. Smith added the comment: New changeset 503de7149d03bdcc671dcbbb5b64f761bb192b4d by Carl Meyer in branch 'master': bpo-40360: Deprecate lib2to3 module in light of PEP 617 (GH-19663) https://github.com/python/cpython/commit/503de7149d03bdcc671dcbbb5b64f761bb192b4d --

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Gregory P. Smith added the comment: Okay,the pending deprecation is in. Keeping open as a reminder to turn that into a real DeprecationWarning in 3.10 after the 3.9 branch is cut. We'll then want to track reminding us to remove it in 3.12. --

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Carl Meyer added the comment: I opened a PR. It deprecates the lib2to3 library to discourage future use of it for Python3, but not the 2to3 tool. This of course means that the lib2to3 module will in practice stick around in the stdlib as long as 2to3 is still bundled with Python. It seems

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Carl Meyer : -- pull_requests: +18987 pull_request: https://github.com/python/cpython/pull/19663 ___ Python tracker ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Eric Snow : -- nosy: +eric.snow ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Batuhan Taskaya : -- nosy: +BTaskaya ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Carl Meyer added the comment: I volunteered in the python-dev thread to write a patch to the docs clarifying future status of lib2to3; happy to include the PendingDeprecationWarning as well. Re linking to alternatives, we want to make sure we link to alternatives that are committed to

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Guido van Rossum added the comment: I am in favor of this. We could promote LibCST, which is based on Parso, which uses a forked version of pgen2 (the parser in lib2to3). I believe one of these could switch to a fork of pegen as its parser, so it will be able to handle new PEG based syntax

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Gregory P. Smith : -- keywords: +patch pull_requests: +18969 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/19645 ___ Python tracker

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

Change by Gregory P. Smith : -- components: +2to3 (2.x to 3.x conversion tool) stage: -> needs patch ___ Python tracker ___ ___

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

New submission from Gregory P. Smith : Based on the PEP 617 acceptance thread on python-dev, lib2to3 is eventually going to run into trouble parsing modern syntax a few releases from now. It would be better off maintained outside of the standard library. It gets used by a lot of things and