HyukjinKwon opened a new pull request, #185: URL: https://github.com/apache/spark-connect-rust/pull/185
### What changes were proposed in this pull request? [SPARK-60958](https://issues.apache.org/jira/browse/SPARK-60958): `SparkSession.builder.remote()` now takes the URL as `url`, as PySpark's `SparkSession.Builder.remote(url)` does, so `remote(url="sc://localhost:15002")` works. The client's one builder also serves `pyspark.sql.connect.session.SparkSession.builder`, whose `remote` in PySpark is `remote(location="sc://localhost")`. So the builder still takes the URL as the keyword `location`, and without a URL it still uses that default. Giving both `url` and `location` raises `TypeError`. ### Why are the changes needed? The parameter was named `location`, so `SparkSession.builder.remote(url=...)` raised `TypeError: SparkSessionBuilder.remote() got an unexpected keyword argument 'url'`. ### Does this PR introduce _any_ user-facing change? Yes, `SparkSession.builder.remote(url=...)` works as in PySpark 4.3.0. ### How was this patch tested? - A new offline test checks the parameters and the error for both keywords. It also starts a mock Connect server and checks that a session built with `remote(url=...)`, and one built with `remote(location=...)`, each send their requests to it. That part runs in a new interpreter, as the session becomes the active one. The test fails without this change. It checks both keywords on the one builder, so it does not run as is with official PySpark, whose two builders each take one of them. - The lib tests and every integration test, e2e included, against a local Spark 4.3.0 server (643 passed), the build of all tests with `wasm-udf`, the offline Python suites (614 passed, 8 live cases skipped), and the live drop-in and Arrow suites (57 passed) and live cases (10 passed). Clippy reports no diagnostics in the changed lines. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
