[
https://issues.apache.org/jira/browse/SPARK-58440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58440:
-----------------------------------
Labels: pull-request-available (was: )
> Sequence codegen puts non-String values into the error message parameter map
> ----------------------------------------------------------------------------
>
> Key: SPARK-58440
> URL: https://issues.apache.org/jira/browse/SPARK-58440
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.1.0, 4.0.0, 4.2.0, 4.3.0, 5.0.0
> Reporter: Yang Jie
> Priority: Major
> Labels: pull-request-available
>
> Under whole-stage codegen, {{sequence()}} raising the "Illegal sequence
> boundaries" error
> produces a {{SparkIllegalArgumentException}} whose message parameters are not
> Strings, even
> though {{SparkThrowable.getMessageParameters()}} declares
> {{java.util.Map<String, String>}}.
> {{Sequence.genSequenceLengthCode}} generates:
> {code:java}
> java.util.Map<String, String> params = new java.util.HashMap<String,
> String>();
> params.put("start", $start);
> params.put("stop", $stop);
> params.put("step", $step);
> throw new
> org.apache.spark.SparkIllegalArgumentException("_LEGACY_ERROR_TEMP_3243",
> params);
> {code}
> {{$start}} and {{$stop}} are numeric expressions, and {{$step}} is either
> numeric or a
> {{CalendarInterval}} reference depending on the sequence implementation.
> Janino, which
> compiles generated code, erases the type arguments and binds these calls to
> {{put(Object, Object)}}, so the values are inserted unconverted. The map then
> flows into
> {{SparkIllegalArgumentException}} and is returned by
> {{getMessageParameters()}} holding
> {{java.lang.Long}} / {{java.lang.Integer}} / {{CalendarInterval}} values. Any
> consumer that
> reads a value as a {{String}} - the declared type - fails with a
> {{ClassCastException}}.
> The rendered message text is unaffected, because parameter substitution calls
> {{toString}}. That is why this has gone unnoticed: only the structured
> parameter map is
> wrong, and this error path has no test coverage.
> Reproducing with the parameter map (codegen path):
> {code:scala}
> val e = intercept[SparkIllegalArgumentException] {
> spark.sql("SELECT sequence(1, 2, 0)").collect()
> }
> e.getMessageParameters // {start=1, stop=2, step=0} with Integer values,
> not String
> {code}
> The fix is to convert in the generated source, i.e.
> {{String.valueOf($start)}}.
> Affected versions: the map-based parameters were introduced by SPARK-46991
> ({{f5b0de07eff}}, replacing {{IllegalArgumentException}} with
> {{SparkIllegalArgumentException}} in catalyst), so this is present from 4.0.0
> onward.
> Verified present on master, branch-4.x, branch-4.2, branch-4.1 and
> branch-4.0. branch-3.5
> is not affected - it still builds the message by string concatenation.
> The interpreted path ({{Sequence.getSequenceLength}}) is a separate matter:
> it throws a
> plain {{IllegalArgumentException}} via {{require}} with no error class and no
> parameters,
> so codegen and interpreted execution are not symmetric here. Reconciling that
> would change
> a user-visible exception type and is out of scope.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
