Oh, this is exciting. I think py2/3 compatible syntax is a good goal due to the tooling (ruff etc) it unlocks and it's relatively benign.
The big exception here is test snippets. Since pypy is still written in py2-only syntax, rpython needs tests that translate py2-only syntax, and hence some test files require snippets that are py2-only. Last time I thought about this, a sensible way forward seemed to be to split these out as explicitly named exceptions to the py3-compatible rule (e.g. `test_yyy_py2.py`). Having attempted this by hand before I'd be happy to assist in reviewing those bite-sized PRs, however I'm not a maintainer. Have a read of https://github.com/pypy/pypy/wiki/RPython-3-considerations if you haven't already (or pass it into your claude agent). Best, On Fri, 25 Sept 2026 at 06:58, Ronny Pfannschmidt < [email protected]> wrote: > Hi everyone, i finished the first milestone i set for myself - which was > rypthon is python3 syntax and it runs/works. now its time to reshuffle and > land those bits so its can be build on. its visible under > https://github.com/RonnyPfannschmidt/pypy/pull/1 - but expect churn to > come up as i prepare eventual actual prs/a pr stack. a key fun bit was > creating a new assert rewrite hook that is rflow compatible plus plays into > my future plans on having annotated/debuggable assertion as part of python > itself as next steps i want to resuffle the pr a bit to be on more merge > friendly comits that should each carry over a enhancement Im also preparing > updates to the ci docker images, but cot.assert may have more churn than > desirable for a while > > i'd appreciate feedback on the order of things and reach some consensus on > how to merge things going forward > > -- Ronny > > ------------------------------ > > report by opus: > > Written by Claude Opus 5.5 via Claude Code for the pypy-dev list; I > prompted it, it did the work, I read it. > > *Milestone 1: all of rpython/ is valid Python 3 syntax* > > The first goal is narrow on purpose: every file under rpython/ parses on > Python 3, while running on Python 2 exactly as before. RPython on a Python > 3 host stays thoroughly broken at this stage: nothing about str/bytes, > removed builtins or the flow space's bytecode handling changes. What the > milestone buys is that from then on, Python 3 work is about behaviour, not > syntax, and tools that parse the source (linters, ast, a Python 3 test > collector) work on the whole tree. > > The milestone is reached on the branch: > https://github.com/RonnyPfannschmidt/pypy/pull/1 (against the fork, for > discussion). Every clean-up commit is valid Python 2 with identical > semantics, so the existing Python 2 CI verifies it, and no call site gains > a sys.version_info check. > > *What it took* > > - Files under rpython/ that do not parse on Python 3: 221 → 0. The > exceptions are three flow space and annotator fixtures that test Python 2 > constructs, which stay Python 2 on purpose. > - Rewritten: 641 print statements, 360 tuple parameters in def and > lambda, 174 L suffixes, 118 octal literals, 17 exec statements, 5 raise > E, v, 4 backticks, <>, has_key, one ur"" literal, and four list > comprehensions over an unparenthesised tuple. > - print x, with a trailing comma needed per-site reading. Python 2 > defers the separating space until the next item and drops it after an item > ending in a newline, while the flow space always inserts it and end= resets > it. Each site was converted by what follows it, so output stays identical > on CPython and in translated code. The one exception is > richards.trace(), which runs only with tracing enabled. > - Some prints parse on both versions but mean different things. print > >> f, x, is a tuple expression on Python 3: one test logger compiled > there and silently logged nothing. And print(a, b) without the > print_function import prints a tuple on Python 2. Both are fixed, and > the second is now counted. > - rpython/tool/py3distance.py counts all of this per category, > including the constructs that only mean something different. A CI job fails > a PR if any count grows. It compares against the PR's base commit, so there > is no stored baseline to conflict over. > - One mechanism fix came with it: pairtype.extendabletype now skips > the names Python 3 adds to the class namespace (__qualname__ and, > since 3.13, __firstlineno__ and friends). With that, all ~275 class > __extend__(X) sites work on a Python 3 host unchanged. > > *Beyond milestone 1 (counted, not started)* > > 753 uses of removed builtins, 202 iteritems()-style calls, 53 imports of > renamed stdlib modules, 13 __metaclass__ attributes. All of these parse > on Python 3 and only fail when run. After them come str/bytes/unicode and > the flow space's bytecode handling. As Matti pointed out, translating with > PyPy as the host stays a hard requirement throughout. > > *Test framework (needed along the way)* > > The vendored pytest 2.9.2 and py 1.4.29 at the repo root block any newer > pytest, and a Python 3 host needs one. The branch removes them, depends on > pytest 4.6.11 (the last to support Python 2), and fixes what that exposed > in rpython/: yield tests (256 tests were being silently skipped), > module-level skips, fixtures called directly, and a duplicate option. > rpython/ collects completely under 4.6 (14391 items). > > pypy/ is not ported: its app-level tests rely on --assert=reinterp, which > pytest 3.0 removed. For now, CI runs pypy/objspace on pytest 2.9.2 from > PyPI (2178 passed, same as main), plus a non-blocking job on 4.6 to track > the gap. package.py still uses py.path, so CI installs the py lib for the > packaging step. > > Asserts in rpython/ tests are rewritten by cot-assert ( > https://pypi.org/project/cot-assert/). It is a small plugin of mine, > started with Claude Opus 5.5, whose rewritten asserts the flow space > accepts, so failing RPython tests get their explanation back. It is > optional and separable from the rest. > > CI: the job that translates and packages PyPy is green. The RPython unit > tests are not fully green yet. Apart from what also fails or flakes on main > (rvmprof test_same_file, the weakref-dict flake, rvmprof TestEnable on > macOS), a handful fail only on the branch: fdlistdir, jitlog, vmprof and > call_release_gil on macOS, a leak check on win64, and one JIT llgraph > test. None of them are in the syntax clean-ups. I am working through them. > > *How it would travel to the py3.x branches* > > I trial-merged the branch into main, py3.11 and py3.12: > > - rpython/ is nearly identical on all three (2–3 differing files), > since main is merged into py3.x regularly. The syntax clean-ups merge > without conflicts there. The three rpython/ test files that do > conflict do so only because main has moved on since the branch point; I > will rebase next. > - The conflicts that matter are in the test infrastructure. The py3.x > branches carry their own changes to the vendored _pytest/py (for > example "fix pytest assertion rewriting on python 2" and "adapt pytest.py > to python3.12"). Removing the vendored copies on main would collide with > those on every merge. > > *A proposed path* > > Land milestone 1 on main in small, independent PRs, each rebased on > current main, and let them reach py3.x through the regular merges: > > 1. The distance tool and its CI check. No code changes, and it stops > new Python-2-only syntax from creeping back in. > 2. The pairtype fix. > 3. The syntax clean-ups, one kind per PR (tuple parameters, octal > literals, L literals, exec, raise, print, ...). Each is valid Python 2 > with the same meaning, so a reviewer checks one rule, not hundreds of > unrelated edits. The trailing-comma prints come as their own PR, since > those were judged one site at a time. > > After that, the test framework as its own discussion, since it is the one > piece that affects the py3.x branches' own pytest changes. Then the runtime > categories, in the same shape. > > *Questions* > > 1. Is "valid Python 3 syntax across rpython/" a milestone you are > happy to have on main, given that it changes nothing about running on > Python 3 yet? > 2. Does "small PRs on main, propagate through the regular merges" work > for you? Or would you rather land in larger batches, or on the py3.x > branches directly? > 3. For the main → py3.x merges: are mechanical sweeps across rpython/ a > burden, or fine given how close rpython/ is on both sides? Is there a > time you would prefer they land, for example right after a merge? > 4. Pytest: one shared pytest for main and py3.x, or a per-branch > vendored copy for now? Would the pytest3.10 branch ( > https://github.com/pypy/pypy/commits/pytest3.10/) be the preferred > base? > 5. What PR size is comfortable to review? > 6. Should anything outside rpython/ be in scope yet? I have kept away > from pypy/, since it diverges between branches. > > Each commit carries a Co-Authored-By trailer naming the model and harness. > > > _______________________________________________ > pypy-dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3//lists/pypy-dev.python.org > Member address: [email protected] >
_______________________________________________ pypy-dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/pypy-dev.python.org Member address: [email protected]
