dongjoon-hyun commented on PR #51681:
URL: https://github.com/apache/spark/pull/51681#issuecomment-3567450312
Hi, @HyukjinKwon and @zhengruifeng .
Unfortunately, according to the CI monitoring result, there is no success
with this PR ('pyarrow==12.0.1' => 'pyarrow>=18.0.0').
-
https://github.com/apache/spark/actions/workflows/build_python_connect35.yml
<img width="861" height="578" alt="Screenshot 2025-11-22 at 19 45 29"
src="https://github.com/user-attachments/assets/2af3329b-d66b-4796-a9a8-7e83e73de117";
/>
It's because PySpark 3.5.x client requires `4.0.0<=PyArrow<13.0.0`.
https://github.com/apache/spark/blob/65c3d1cb18c45528d8090ac905d87a8dcd779aa7/python/docs/source/getting_started/install.rst?plain=1#L160
The root cause is the breaking change of PyArrow 13.0.0.
- #42920
- #45553
Given that, let me revert this test infra commit inevitably because this is
misleading. Sorry about this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
