DB Tsai created SPARK-60596:
-------------------------------
Summary: Upgrade PyO3 to 0.30.1 in the Spark Connect Rust client
Key: SPARK-60596
URL: https://issues.apache.org/jira/browse/SPARK-60596
Project: Spark
Issue Type: Dependency upgrade
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: DB Tsai
Upgrade the Spark Connect Rust Python extension from PyO3 0.28.3 to 0.30.1 in a
standalone dependency change.
Scope:
* Bump the workspace PyO3 dependency and the five PyO3 lockfile entries, adding
the required Syn 3 dependency.
* Declare Rust 1.85 for pyspark-rs to match PyO3 0.30.1's minimum compiler
requirement. Leave the workspace default unchanged.
* Preserve extension-module and abi3-py39, and remove an outdated
version-specific comment.
This separates the dependency upgrade from SPARK-60586 / spark-connect-rust PR
#103. It does not change the Arrow handoff, scalar conversion, or Python
wrapper implementation.
Validation so far: optimized baseline and upgraded wheels built offline; 388
core unit tests passed; 273 existing offline Python tests passed on each of
Python 3.9 with PyArrow 10 and Python 3.12 with PyArrow 26; integration tests
compiled with and without wasm-udf; official collection parity passed (15
passed, 1 skipped); formatting passed. Public API names are unchanged. Full
official parity and a Rust 1.85 build have not been verified locally. Workspace
Clippy errors also reproduce on the unchanged upstream baseline.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
