Brij Raj Kishore created SPARK-59872:
----------------------------------------
Summary: [PYTHON] Raise structured PySparkRuntimeError for
PermissionError and other OSError subclasses when reading Protobuf descriptor
file
Key: SPARK-59872
URL: https://issues.apache.org/jira/browse/SPARK-59872
Project: Spark
Issue Type: Improvement
Components: PySpark
Affects Versions: 4.4.0
Reporter: Brij Raj Kishore
Follow-up to SPARK-43847 (PR #56174).
_read_descriptor_set_file() in pyspark.sql.protobuf.functions currently maps
only FileNotFoundError to the structured PROTOBUF_DESCRIPTOR_FILE_NOT_FOUND
error condition. Other OSError subclasses (PermissionError, IsADirectoryError,
etc.) propagate as raw Python exceptions.
This ticket adds a new PROTOBUF_DESCRIPTOR_FILE_READ_FAILED error condition
and maps those remaining cases to a structured PySparkRuntimeError, consistent
with the approach taken for FileNotFoundError.
The Scala side currently maps these to CANNOT_PARSE_PROTOBUF_DESCRIPTOR, which
nchammas noted is not ideal — a Python-specific error condition is more
accurate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
