+1
2026年9月2日(水) 16:43 Holden Karau <[email protected]>: > +1 > > > Twitter: https://twitter.com/holdenkarau > Fight Health Insurance: https://www.fighthealthinsurance.com/ > <https://www.fighthealthinsurance.com/?q=hk_email> > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > Pronouns: she/her > > On Tue, Sep 1, 2026 at 9:15 PM Dongjoon Hyun <[email protected]> wrote: > >> +1 >> >> Dongjoon. >> >> On 2026/09/01 22:08:01 Hyukjin Kwon wrote: >> > Starting with my own +1. >> > >> > PS: we have a discussion in the dev mailing list, and I am sync'ed with >> the >> > original authors. >> > >> > On Wed, 2 Sept 2026 at 07:07, Hyukjin Kwon <[email protected]> >> wrote: >> > >> > > Please vote on releasing the following candidate as Apache Spark >> Connect >> > > Rust >> > > Client 4.2.0 RC2. >> > > >> > > This is the *first* release of the ground-up Rust rewrite of the Spark >> > > Connect >> > > Python client (published to PyPI as pyspark-client-rust): a drop-in >> > > replacement >> > > for pyspark-client <https://pypi.org/project/pyspark-client/>, >> backed by >> > > a native Rust core instead of py4j/grpcio. >> > > >> > > The vote is open until Thursday, September 7, 2026, 5:00 PM PDT and >> passes >> > > if a majority of +1 PMC votes are cast, with a minimum of >> > > 3 +1 votes. >> > > >> > > [ ] +1 Release this package as Apache Spark Connect Rust Client 4.2.0 >> > > [ ] -1 Do not release this package because … >> > > >> > > TAG: >> > > https://github.com/apache/spark-connect-rust/releases/tag/v4.2.0-rc2 >> > > (commit: ce1fa08) >> > > >> > > RELEASE FILES (wheels + sdist, built and attached by GitHub Actions): >> > > https://github.com/apache/spark-connect-rust/releases/tag/v4.2.0-rc2 >> > > >> > > DOCUMENTATION: >> > > https://apache.github.io/spark-connect-rust/ >> > > >> > > LIST OF ISSUES: >> > > https://issues.apache.org/jira/projects/SPARK/versions/12357484 >> > > HOW TO TRY IT — PYTHON (pip) >> > > >> > > The release attaches abi3 wheels (Linux x86_64/aarch64, macOS >> universal2, >> > > Windows x86_64; CPython 3.9+) plus a source distribution. Grab the >> wheel >> > > for >> > > your platform from the release page and install it directly, e.g.: >> > > >> > > pip install >> https://github.com/apache/spark-connect-rust/releases/download/v4.2.0-rc2/pyspark_client_rust-4.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl >> > > >> > > Then use it exactly like PySpark. See if your existing workloads work >> with >> > > this. >> > > HOW TO TEST IT — RUST (cargo) >> > > >> > > The Rust crates are intentionally NOT on crates.io for a release >> candidate >> > > (crates.io is immutable; the crates are published there only when the >> > > release is >> > > finalized). Test against the tagged source instead: >> > > >> > > # A) Pin a git dependency to the RC tag in your project's >> Cargo.toml: >> > > [dependencies] >> > > apache-spark-connect = { git = " >> https://github.com/apache/spark-connect-rust";, tag = "v4.2.0-rc2" } >> > > >> > > # B) Or build and run the crates' own tests from the tag: >> > > git clone --branch v4.2.0-rc2 >> https://github.com/apache/spark-connect-rust >> > > cd spark-connect-rust >> > > cargo build --release >> > > cargo test >> > > >> > > HIGHLIGHTS >> > > >> > > - *Drop-in PySpark Connect Python API*, verified against real >> PySpark >> > > 4.2.0 two >> > > ways: the official PySpark test suite runs against this client, >> and a >> > > behavioral API-parity sweep compares the Python API surface >> (functions >> > > and >> > > DataFrame / Column / SparkSession / GroupedData / Window methods) >> > > call-by-call >> > > to PySpark. >> > > - Spark Connect Rust API *full* coverage, including *Rust UDF*s >> > > (docs: https://apache.github.io/spark-connect-rust/). >> > > - *Native Rust transport* via tonic; cloudpickle vendored to match >> > > upstream, so >> > > no grpcio/py4j at runtime. and *native type conversion.* >> > > >> > > WHY DOES THE VERSION START AT 4.2.0? >> > > >> > > Even though this is the first release of the rewrite, the version >> starts at >> > > 4.2.0 to track the Apache Spark / PySpark version it targets. This (1) >> > > removes >> > > any ambiguity about which Spark version a given client is compatible >> with, >> > > and >> > > (2) lets PySpark users pip install pyspark-client-rust== exactly >> > > as they would pyspark-client, keeping the same versioning across both. >> > > >> > > Thanks, >> > > Hyukjin Kwon >> > > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: [email protected] >> >>
