We didn't have such a SPIP or official vote when dropping the Python 3.9 and Python 3.8. I think we need to mention it in the migration guide https://apache.github.io/spark/api/python/migration_guide/pyspark_upgrade.html
On Tue, Mar 10, 2026 at 3:07 AM Tian Gao via dev <[email protected]> wrote: > Okay, it sounds like people support this motion. My question is: Do I need > to draft an SPIP for this and start an official vote? Maybe deprecating an > once supported platform requires some official documentation? > > Tian > > On Mon, Mar 9, 2026 at 10:34 AM Holden Karau <[email protected]> > wrote: > >> +1 >> >> Twitter: https://twitter.com/holdenkarau >> Fight Health Insurance: https://www.fighthealthinsurance.com/ >> <https://www.fighthealthinsurance.com/?q=hk_email> >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> Pronouns: she/her >> >> >> On Mon, Mar 9, 2026 at 8:48 AM Dongjoon Hyun <[email protected]> wrote: >> >>> +1 >>> >>> Dongjoon >>> >>> On Sun, Mar 8, 2026 at 23:53 Yang Jie <[email protected]> wrote: >>> >>>> +1 >>>> >>>> On 2026/03/09 04:40:54 Reynold Xin via dev wrote: >>>> > +1 >>>> > >>>> > On Sun, Mar 8, 2026 at 7:20 PM Hyukjin Kwon <[email protected]> >>>> wrote: >>>> > >>>> > > +1 >>>> > > >>>> > > On Mon, 9 Mar 2026 at 10:37, Ruifeng Zheng <[email protected]> >>>> wrote: >>>> > > >>>> > >> +1 >>>> > >> >>>> > >> I remember PyArrow doesn't support PyPy, either. >>>> > >> >>>> > >> Because of the missing support of these dependencies, the test >>>> coverage >>>> > >> of PyPy is low, it basically only tests Core and SQL. >>>> > >> Tests for Connect, ML, Structured Streaming and Pandas API are >>>> always >>>> > >> skipped in CI. >>>> > >> >>>> > >> On Mon, Mar 9, 2026 at 4:34 AM Tian Gao via dev < >>>> [email protected]> >>>> > >> wrote: >>>> > >> >>>> > >>> We claim to support pypy, but as far as I know, no one is really >>>> > >>> maintaining it. pypy3.11 CI has been failing for a long time, and >>>> we've >>>> > >>> just ignored it. >>>> > >>> >>>> > >>> numpy has dropped support for pypy recently - >>>> > >>> https://github.com/numpy/numpy/pull/30764 because pypy itself is >>>> not >>>> > >>> well maintained. They have not announced abandonment, but there >>>> have only >>>> > >>> been 7 commits this year. The latest python version they support >>>> is 3.11 >>>> > >>> (CPython is 3.14 now). There was a verbal plan to support 3.12 >>>> but the >>>> > >>> progress is unclear. >>>> > >>> >>>> > >>> We could've used the CI resources to test other much more common >>>> > >>> platform/version combinations for spark. >>>> > >>> >>>> > >>> Overall: >>>> > >>> * pypy seems to be dying >>>> > >>> * few people are really using it >>>> > >>> * we do not care about it enough to fix the CI >>>> > >>> * we can have more resources on important use cases. >>>> > >>> * if numpy dropped support, we will lose a lot of the use cases >>>> anyway >>>> > >>> >>>> > >>> How do we feel about this? Do we really have a reason to keep it >>>> > >>> supported? >>>> > >>> >>>> > >>> Tian Gao >>>> > >>> >>>> > >> >>>> > >> >>>> > >> -- >>>> > >> Ruifeng Zheng >>>> > >> E-mail: [email protected] >>>> > >> >>>> > > >>>> > >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: [email protected] >>>> >>>> -- Ruifeng Zheng E-mail: [email protected]
