+1 I remember PyArrow doesn't support PyPy, either.
Because of the missing support of these dependencies, the test coverage of PyPy is low, it basically only tests Core and SQL. Tests for Connect, ML, Structured Streaming and Pandas API are always skipped in CI. On Mon, Mar 9, 2026 at 4:34 AM Tian Gao via dev <[email protected]> wrote: > We claim to support pypy, but as far as I know, no one is really > maintaining it. pypy3.11 CI has been failing for a long time, and we've > just ignored it. > > numpy has dropped support for pypy recently - > https://github.com/numpy/numpy/pull/30764 because pypy itself is not well > maintained. They have not announced abandonment, but there have only been 7 > commits this year. The latest python version they support is 3.11 (CPython > is 3.14 now). There was a verbal plan to support 3.12 but the progress is > unclear. > > We could've used the CI resources to test other much more common > platform/version combinations for spark. > > Overall: > * pypy seems to be dying > * few people are really using it > * we do not care about it enough to fix the CI > * we can have more resources on important use cases. > * if numpy dropped support, we will lose a lot of the use cases anyway > > How do we feel about this? Do we really have a reason to keep it supported? > > Tian Gao > -- Ruifeng Zheng E-mail: [email protected]
