Hi all, I'd like to ask whether the community would consider cancelling RC1 and cutting an RC2 that includes GH-50326 / PR #50327 (making Array.to_pylist convert without per-element Scalars; reworked per Antoine's review and awaiting another pass).
I realize this doesn't meet the usual bar for sinking an RC — it is a long-standing inefficiency, not a regression in PyArrow itself. The reason I'm asking is timing on the Spark side: Spark 4.2, already at RC, switches regular Python UDFs to Arrow serialization by default, and that path materializes Arrow columns to Python objects through to_pylist(). With the current implementation this shows up to Spark users as a performance regression on upgrade (e.g. to_pylist on a 2M-row list<string> column takes 1.93s vs 0.34s with the fix; several times slower than the old pickle path on nested types end-to-end). Arrow 25.0.0 is the last release that can plausibly reach users while Spark 4.2 rolls out — if the fix slips to the next Arrow release, early Spark 4.2 adopters would live with this regression for months. A small follow-up is also ready and stacked on it: GH-50429 / PR #50430 extends the same mechanism to to_pylist(maps_as_pydicts=...), converting maps to dicts about 21x faster (2.20s → 0.10s for 1M rows) — the map-to-dict form is exactly what Spark's UDF path consumes. I'd be happy to have either just #50327 or both in RC2, whichever the community is comfortable with. If the community prefers to proceed with RC1 as-is, that's completely understandable — I'll target the next release and we'll carry interim workarounds on the Spark side. But if there's openness to an RC2, I'll do my best to get the reviews wrapped up quickly. Thanks. On Mon, Jul 6, 2026 at 1:27 AM Raúl Cumplido <[email protected]> wrote: > Hi, > > I would like to propose the following release candidate (RC1) of Apache > Arrow version 25.0.0. This is a release consisting of 220 > resolved GitHub issues[1]. > > This release candidate is based on commit: > 59bea6ec485e7fe351d1aa6753f964f6a6bc353a [2] > > The source release rc1 is hosted at [3]. > The binary artifacts are hosted at [4][5][6][7][8][9]. > The changelog is located at [10]. > > Please download, verify checksums and signatures, run the unit tests, > and vote on the release. See [11] for the SHA-512 checksum for this RC and > [12] > for how to validate a release candidate. > > See also a verification result on GitHub pull request [13]. > > The vote will be open for at least 72 hours. > > [ ] +1 Release this as Apache Arrow 25.0.0 > [ ] +0 > [ ] -1 Do not release this as Apache Arrow 25.0.0 because... > > [1]: > https://github.com/apache/arrow/issues?q=is%3Aissue+milestone%3A25.0.0+is%3Aclosed > [2]: > https://github.com/apache/arrow/tree/59bea6ec485e7fe351d1aa6753f964f6a6bc353a > [3]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-25.0.0-rc1 > [4]: https://packages.apache.org/artifactory/arrow/almalinux-rc/ > [5]: https://packages.apache.org/artifactory/arrow/amazon-linux-rc/ > [6]: https://packages.apache.org/artifactory/arrow/centos-rc/ > [7]: https://packages.apache.org/artifactory/arrow/debian-rc/ > [8]: https://packages.apache.org/artifactory/arrow/ubuntu-rc/ > [9]: https://github.com/apache/arrow/releases/tag/apache-arrow-25.0.0-rc1 > [10]: > https://github.com/apache/arrow/blob/59bea6ec485e7fe351d1aa6753f964f6a6bc353a/CHANGELOG.md > [11]: > 9c7a91eef5177b0339f0479397afcc455f25af4d0cb07cd5ce0dcbdd5bba390ef2310cd114b6769c13f93e7199d58a895b2d680cca997afb6ebdb8e7dabf5b4e > [12]: https://arrow.apache.org/docs/developers/release_verification.html > [13]: https://github.com/apache/arrow/pull/50357 >
