Hyukjin Kwon created SPARK-61086:
------------------------------------
Summary: Validate the startup URLs of SparkSession.Builder, and
name its class Builder, in the Python client as PySpark does
Key: SPARK-61086
URL: https://issues.apache.org/jira/browse/SPARK-61086
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
In PySpark Connect, {{SparkSession.builder}} is a {{SparkSession.Builder}}, and
its {{config}}, {{master}} and {{remote}} validate the startup URLs: a master
({{spark.master}} or {{MASTER}}) with a remote ({{spark.remote}} or
{{SPARK_REMOTE}}) raises the {{PySparkRuntimeError}}
{{CANNOT_CONFIGURE_SPARK_CONNECT_MASTER}}, and a remote other than
{{SPARK_REMOTE}} with {{SPARK_LOCAL_REMOTE}} raises
{{CANNOT_CONFIGURE_SPARK_CONNECT}}.
The Rust-backed Python client names the class of the builder
{{SparkSessionBuilder}}, and takes a master with a remote. Validate them as
PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
