[
https://issues.apache.org/jira/browse/SPARK-60418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60418:
-----------------------------------
Labels: pull-request-available (was: )
> Subexpression elimination evaluates a short-circuited IN list element
> ---------------------------------------------------------------------
>
> Key: SPARK-60418
> URL: https://issues.apache.org/jira/browse/SPARK-60418
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: sepuri sai krishna
> Priority: Minor
> Labels: pull-request-available
>
> {{In}} stops at the first list element that matches, in both {{eval}} and
> generated code, but a later element is still evaluated ahead of the comparison
> when it is a common subexpression. A query that should return rows then fails.
> ANSI mode, default configuration:
> {code:sql}
> CREATE OR REPLACE TEMP VIEW t AS
> SELECT cast(id as int) AS b, cast(2147483647 - id as int) AS y FROM
> range(1);
> SELECT b IN (b, y + 1), b IN (b, y + 1, 7) FROM t;
> -- b matches the first element of both lists, so y + 1 is never needed
> -- expected: true, true
> -- actual: [ARITHMETIC_OVERFLOW] overflow
> {code}
> Either {{IN}} on its own returns {{true}}, and so does the query with
> {{spark.sql.subexpressionElimination.enabled}} set to {{false}}, so the error
> comes from the elimination rather than from the query. It fails with
> whole-stage
> codegen both on and off. A list of literals is not affected.
> {{spark.sql.subexpressionElimination.skipForShortcutExpr}}, which exists to
> stop
> elimination from evaluating short-circuited operands, does not help here:
> with it
> set to {{true}} the same shape over {{AND}} stops failing, while the {{IN}}
> query
> above still fails. {{In}} short-circuits the same way, and is already listed
> alongside {{And}} and {{Or}} as conditionally evaluated in
> {{OptimizeCsvJsonExprs}}.
> Verified on master at 8d5b30a06f6.
> ---
> ## Why only one
> The try_* candidate does not hold up. {{try_subtract(cast(s as date), cast(s
> as
> date))}} does raise with elimination on and return NULL with it off, but NULL
> is
> not what try_* promises:
> - the numeric path, {{try_subtract(cast(s as int), 1)}}, raises the operand's
> CAST_INVALID_INPUT in every mode, so try_* does not swallow operand errors
> by
> design;
> - what the docs promise is NULL on the operation's own overflow, and
> {{try_add(date '2020-01-01', make_ym_interval(178956970, 0))}} returns NULL
> in
> every mode, elimination on or off;
> - the non-numeric path swallows operand errors only incidentally, because
> {{TryEval}} wraps the whole subtree, and the code carries a
> "TODO: support TRY eval mode on datetime arithmetic expressions" there.
> So with elimination on, the non-numeric path becomes consistent with the
> numeric
> one. Filing it as "try_* raises instead of returning NULL" would be arguing
> for
> the anomalous behaviour.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
