Subhramit Basu Bhowmick created SPARK-58945:
-----------------------------------------------
Summary: Mismatched `messageParameters` keys cause
`INTERNAL_ERROR` instead of the intended error
Key: SPARK-58945
URL: https://issues.apache.org/jira/browse/SPARK-58945
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Subhramit Basu Bhowmick
Several error constructors pass {{messageParameters}} keys that do not match
the placeholders declared in {{{}error-conditions.json{}}}.
Spark's {{StringSubstitutor}} defaults to {{enableUndefinedVariableException =
true}}
({{{}common/utils/src/main/scala/org/apache/spark/StringSubstitutor.scala:29{}}}),
so an unresolved placeholder throws {{{}IllegalArgumentException{}}}, which
{{ErrorClassesJSONReader}} converts into {{{}SparkException.internalError{}}}.
The message rendering is eager - {{SparkThrowableHelper.getMessage}} is called
from the exception constructor ({{{}SparkException.scala:288{}}}) - so the
intended exception is never thrown. The user receives {{INTERNAL_ERROR}}
("Undefined error message parameter for error class: ...") in place of the
actual diagnosis.
h3. Repro
{code:scala}
spark.range(1).write.option("extension", "12").csv(path)
{code}
Expected: {{INVALID_PARAMETER_VALUE.EXTENSION}}
Actual: {{INTERNAL_ERROR}}
h3. Affected sites
Verified against {{{}error-conditions.json{}}}, ordered by reachability:
{noformat}
QueryExecutionErrors.scala:3214 INVALID_PARAMETER_VALUE.EXTENSION
expects invalidValue; passes fileExtension + acceptable
reached from CSVOptions.scala:128 via the "extension" write option
QueryCompilationErrors.scala:4677 _LEGACY_ERROR_TEMP_2450
expects clazz; passes invalidClass
reached from HiveSessionStateBuilder.scala:253
QueryExecutionErrors.scala:3242 INVALID_WRITER_COMMIT_MESSAGE
expects detail; passes details
reached from PythonBatchWriterFactory.scala:58,62
H2Dialect.scala:230 TABLE_OR_VIEW_NOT_FOUND
missing searchPath
FileSourceStrategy.scala:291 _LEGACY_ERROR_TEMP_3070
expects colName/internalName; passes field
StateStoreErrors.scala:399 STATE_STORE_COLUMN_FAMILY_SCHEMA_INCOMPATIBLE
expects colFamilyName/oldSchema/newSchema
passes columnFamilyName/oldColumnFamilySchema/newColumnFamilySchema
NOTE: the stateStoreColumnFamilyMismatch factory currently has no
callers, so this one is latent rather than user-facing today.
{noformat}
These survived because none of these error paths are exercised by tests; any
test constructing these exceptions would fail at construction.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
