Hyukjin Kwon created SPARK-61083:
------------------------------------
Summary: Raise the errors of PySpark for the tags, the table name,
the paths of copyFromLocalToFs and the JVM attributes of a session in the
Python client
Key: SPARK-61083
URL: https://issues.apache.org/jira/browse/SPARK-61083
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
In PySpark Connect, the methods of a session raise these errors for their
arguments:
* {{addTag}} and {{removeTag}} raise {{CANNOT_BE_NONE}} for None,
{{VALUE_ALLOWED}} for a tag of a comma, and {{VALUE_NOT_NON_EMPTY_STR}} for an
empty tag, of the {{arg_name}} "Spark Connect tag".
* {{table}} raises {{NOT_EXPECTED_TYPE}} for a name that is not a str.
* {{copyFromLocalToFs}} raises {{NO_SCHEMA_AND_DRIVER_DEFAULT_SCHEME}} for a
destination of a scheme, and an {{AssertionError}} ("local path must be a file
path.") for a local path that is not a file.
* {{_jsc}}, {{_jconf}}, {{_jvm}}, {{_jsparkSession}} and {{sparkContext}} raise
the {{PySparkAttributeError}} {{JVM_ATTRIBUTE_NOT_SUPPORTED}}.
The Rust-backed Python client raises {{INVALID_TAG}} for the tags of
{{addTag}}, takes any tag in {{removeTag}}, raises a plain {{TypeError}} of
{{table(1)}}, a {{PySparkRuntimeError}} of a missing local file, and a plain
{{AttributeError}} of {{sparkContext}}. Raise them as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
