L. C. Hsieh created SPARK-61663:
-----------------------------------

             Summary: Let each expression declare whether it is total instead 
of the allowlist in ExprUtils.isTotalOperator
                 Key: SPARK-61663
                 URL: https://issues.apache.org/jira/browse/SPARK-61663
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 5.0.0
            Reporter: L. C. Hsieh


ExprUtils.isTotalOperator decides whether an expression can raise an error by 
itself with one hand-maintained pattern match over expression classes. 
ExprUtils.canEvaluateUnconditionally, ConstantPropagation and the optimizer and 
execution rules that call them all go through it.

Two things make the list hard to maintain:
* An expression that is missing from the list is only left out of some 
optimizations, so nothing flags the omission, and the list lives far from the 
expression it describes.
* The cases for Cast and the arithmetic expressions depend on the input types 
and the eval mode, and their results rely on the order of the case clauses.

This proposes to let each expression declare it:
* Expression.isTotal is the declaration, false by default.
* An expression that is always total mixes in the existing marker trait 
NoThrow, which fixes isTotal to true.
* Cast, Add, Subtract, Multiply, Divide, IntegralDivide, Remainder, Pmod, 
UnaryMinus and Abs override isTotal by their input types and eval mode, each as 
a union of independent conditions.
* ExprUtils.isTotalOperator(e) becomes e.isTotal, and its callers do not change.

This is a refactoring with no behavior change: the set of expressions 
isTotalOperator accepts stays the same in every eval mode.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to