Ganesha S created SPARK-58218:
---------------------------------
Summary: [SQL] Replace generic RuntimeException with typed
exceptions in AvroDeserializer
Key: SPARK-58218
URL: https://issues.apache.org/jira/browse/SPARK-58218
Project: Spark
Issue Type: Task
Components: SQL
Affects Versions: 4.2.0
Reporter: Ganesha S
`AvroDeserializer` throws three generic `java.lang.RuntimeExceptions` where the
rest of the class, and the rest of Spark, use typed exceptions. This makes the
failures uncatchable by error class, gives them no SQLSTATE, and, for one of
the sites, is inconsistent with the sibling cases in the same method.
The three sites on the current master
(sql/core/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala):
- Line 278 — a value that is not a valid Avro binary during BYTES -> BinaryType
conversion:
{code:scala}
case other =>
throw new RuntimeException(errorPrefix + s"$other is not a valid avro
binary.")
{code}
Every other type-mismatch case in the same newWriter method (lines 187, 202,
212, 224, 233) throws IncompatibleSchemaException(errorPrefix + ...). Only this
one uses a bare RuntimeException.
- Line 332 — a null element read into a non-nullable array element type:
{code:scala}
throw new RuntimeException(
s"Array value at path ${toFieldStr(avroElementPath)} is not allowed to be
null")
{code}
- Line 364 — a null value read into a non-nullable map value type:
{code:scala}
throw new RuntimeException(
s"Map value at path ${toFieldStr(avroPath :+ "value")} is not allowed to be
null")
{code}
*Why this is a problem*
Spark has largely migrated runtime failures to the error-class framework
(SparkRuntimeException, SparkException.internalError, named error conditions
with SQLSTATE). Generic RuntimeExceptions at these sites:
- cannot be caught or matched on an error condition by callers,
- carry no SQLSTATE, and
- (line 278) are inconsistent with the five sibling cases in the same method
that already use IncompatibleSchemaException.
Notably, the write path already has a typed, catalogued error for the analogous
situation: AVRO_CANNOT_WRITE_NULL_FIELD (SQLSTATE 22004). The read path has no
equivalent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
