Yang Jie created SPARK-59579:
--------------------------------
Summary: Keep the conditional-branch guards after the
subexpression elimination shortcut peel
Key: SPARK-59579
URL: https://issues.apache.org/jira/browse/SPARK-59579
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Yang Jie
{{EquivalentExpressions.childrenToRecurse}} stops the descent for expressions
whose children must not be evaluated ahead of time: {{CodegenFallback}},
{{ConditionalExpression}} (only {{alwaysEvaluatedInputs}} are recursed into)
and {{HigherOrderFunction}}.
When {{spark.sql.subexpressionElimination.skipForShortcutExpr}} is enabled,
{{skipForShortcut}} first peels the leading {{And}}/{{Or}} operands to reach
the one operand that is always evaluated. {{And}}/{{Or}} are not
{{ConditionalExpression}}s, so the peel walks past those cases, and the code
then took {{peeled.children}} directly -- recursing into every child of the
expression the peel landed on, conditional branches included.
Repro, with ANSI mode on and both
{{spark.sql.subexpressionElimination.enabled}} and
{{spark.sql.subexpressionElimination.skipForShortcutExpr}} set to true:
{code:sql}
select (case when id = 0 then false else (1 / id + 1 / id) > 0 end) and id >= 0
from range(0, 1, 1, 1)
{code}
{{1 / id}} is repeated inside one branch body only, so it is shared with no
other branch and should be evaluated only when that branch runs. For id = 0 the
other branch runs. Instead {{1 / id}} is collected as a common subexpression,
hoisted to the top of the projection and evaluated for id = 0, raising
DIVIDE_BY_ZERO.
The fix is to compute the always-evaluated operand once and ask both
{{childrenToRecurse}} and {{commonChildrenToRecurse}} about that operand. Only
the {{skipForShortcutExpr}} path changes, and the config default (false) stays.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
