L. C. Hsieh created SPARK-61556:
-----------------------------------

             Summary: Generate FilterExec predicate code with and without 
subexpression elimination from one implementation
                 Key: SPARK-61556
                 URL: https://issues.apache.org/jira/browse/SPARK-61556
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 4.4.0
            Reporter: L. C. Hsieh


{{FilterExec.doConsume}} re-implements, for the conjunctions it eliminates 
common subexpressions from, the algorithm of 
{{GeneratePredicateHelper.generatePredicateCode}} that orders the conjuncts and 
the IsNotNull checks in the generated code. The two copies are kept in sync 
only by comments ("Any change to this algorithm must be mirrored there (and 
vice versa)"), and earlier fixes such as SPARK-51356 and SPARK-60315 had to be 
applied to both.

This is to have a single implementation: {{generatePredicateCode}} takes the 
states of the common subexpressions, evaluates each right before the first 
conjunct that references it, after the IsNotNull checks of that conjunct, and 
generates the checks without the states. {{FilterExec.doConsume}} keeps the 
decision to eliminate, the evaluation of the input variables and the creation 
of the states.

No behavior change is intended: the generated code without subexpression 
elimination stays the same, and with it the code differs in whitespace only.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to