[
https://issues.apache.org/jira/browse/SPARK-58218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58218:
-----------------------------------
Labels: pull-request-available (was: )
> [SQL] Replace generic RuntimeException with typed exceptions in
> AvroDeserializer
> --------------------------------------------------------------------------------
>
> Key: SPARK-58218
> URL: https://issues.apache.org/jira/browse/SPARK-58218
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Ganesha S
> Priority: Major
> Labels: pull-request-available
>
> `AvroDeserializer` throws three generic `java.lang.RuntimeExceptions` where
> the rest of the class, and the rest of Spark, use typed exceptions. This
> makes the failures uncatchable by error class, gives them no SQLSTATE, and,
> for one of the sites, is inconsistent with the sibling cases in the same
> method.
> The three sites on the current master
> (sql/core/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala):
> - Line 278 — a value that is not a valid Avro binary during BYTES ->
> BinaryType conversion:
> {code:scala}
> case other =>
> throw new RuntimeException(errorPrefix + s"$other is not a valid avro
> binary.")
> {code}
> Every other type-mismatch case in the same newWriter method (lines 187, 202,
> 212, 224, 233) throws IncompatibleSchemaException(errorPrefix + ...). Only
> this one uses a bare RuntimeException.
> - Line 332 — a null element read into a non-nullable array element type:
> {code:scala}
> throw new RuntimeException(
> s"Array value at path ${toFieldStr(avroElementPath)} is not allowed to be
> null")
> {code}
> - Line 364 — a null value read into a non-nullable map value type:
> {code:scala}
> throw new RuntimeException(
> s"Map value at path ${toFieldStr(avroPath :+ "value")} is not allowed to be
> null")
> {code}
> *Why this is a problem*
> Spark has largely migrated runtime failures to the error-class framework
> (SparkRuntimeException, SparkException.internalError, named error conditions
> with SQLSTATE). Generic RuntimeExceptions at these sites:
> - cannot be caught or matched on an error condition by callers,
> - carry no SQLSTATE, and
> - (line 278) are inconsistent with the five sibling cases in the same method
> that already use IncompatibleSchemaException.
> Notably, the write path already has a typed, catalogued error for the
> analogous situation: AVRO_CANNOT_WRITE_NULL_FIELD (SQLSTATE 22004). The read
> path has no equivalent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
