[
https://issues.apache.org/jira/browse/SPARK-58945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58945:
-----------------------------------
Labels: pull-request-available (was: )
> Mismatched `messageParameters` keys cause `INTERNAL_ERROR` instead of the
> intended error
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-58945
> URL: https://issues.apache.org/jira/browse/SPARK-58945
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Subhramit Basu Bhowmick
> Priority: Minor
> Labels: pull-request-available
>
> Several Spark error paths are broken by mismatches between
> {{messageParameters}} keys and the placeholders declared in
> {{error-conditions.json}}. In addition, one legacy error class has a message
> template duplicated from its neighbour. These either surface
> {{INTERNAL_ERROR}} instead of the intended error condition, or report the
> wrong message text.
> h3. Mechanism
> Spark's {{StringSubstitutor}} defaults to {{enableUndefinedVariableException
> = true}}
> ({{common/utils/src/main/scala/org/apache/spark/StringSubstitutor.scala:29}}),
> so an unresolved placeholder throws {{IllegalArgumentException}}, which
> {{ErrorClassesJSONReader}} converts into {{SparkException.internalError}}.
> Message rendering is eager: the relevant exception constructors call
> {{SparkThrowableHelper.getMessage}} during construction (for example
> {{common/utils/src/main/scala/org/apache/spark/SparkException.scala:288}}).
> When substitution fails, the intended exception instance is never created,
> and the user receives {{INTERNAL_ERROR}} ("Undefined error message parameter
> for error class: ...") in place of the actual diagnosis.
> h3. Repro
> {code:scala}
> spark.range(1).write.option("extension", "12").csv(path)
> {code}
> Expected: {{INVALID_PARAMETER_VALUE.EXTENSION}}
> Actual: {{INTERNAL_ERROR}}
> h3. Parameter key mismatches
> Verified against {{error-conditions.json}}, ordered by reachability:
> {noformat}
> QueryExecutionErrors.scala:3214 INVALID_PARAMETER_VALUE.EXTENSION
> expects invalidValue; passes fileExtension + acceptable
> reached from CSVOptions.scala:128 via the "extension" write option
> QueryCompilationErrors.scala:4677 _LEGACY_ERROR_TEMP_2450
> expects clazz; passes invalidClass
> reached from HiveSessionStateBuilder.scala:253
> QueryExecutionErrors.scala:3242 INVALID_WRITER_COMMIT_MESSAGE
> expects detail; passes details
> reached from PythonBatchWriterFactory.scala:58,62
> H2Dialect.scala:230 TABLE_OR_VIEW_NOT_FOUND
> missing searchPath; classifyException receives only pre-rendered
> strings from JDBCTableCatalog.scala:104-106, so no Spark-side search
> path exists at that point
> StateStoreErrors.scala:399
> STATE_STORE_COLUMN_FAMILY_SCHEMA_INCOMPATIBLE
> expects colFamilyName/oldSchema/newSchema
> passes columnFamilyName/oldColumnFamilySchema/newColumnFamilySchema
> NOTE: the stateStoreColumnFamilyMismatch factory currently has no
> callers, so this one is latent rather than user-facing today.
> {noformat}
> h3. Duplicated message template
> {{_LEGACY_ERROR_TEMP_3069}} and {{_LEGACY_ERROR_TEMP_3070}} have
> byte-identical message templates. 3069 ({{FileSourceStrategy.scala:271}}) is
> the reserved-column-name collision and its text is correct. 3070
> ({{FileSourceStrategy.scala:291}}) is the unrecognized-metadata-field
> fallback and inherited 3069's text during the error-class migration in
> SPARK-46351.
> Here the call site is not at fault - its {{Map("field" -> field.toString)}}
> was already correct. The template is the bug, and 3070 currently reports a
> reserved-column-name collision that never occurred. Before SPARK-46351 this
> branch threw {{s"Unrecognized file metadata field: $field"}}.
> h3. Why these survived
> None of these error paths are exercised by tests. Because message rendering
> is eager, any test asserting the intended error condition on an affected path
> would fail during exception construction.
> Note the site list is a lower bound. The sweep covered
> {{errorClass}}/{{messageParameters}} literal pairs in {{sql/}}, {{core/}},
> and {{mllib/}}, and skips non-literal parameter maps, non-literal error class
> arguments, and {{.java}} sources.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
