[ 
https://issues.apache.org/jira/browse/SPARK-61556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

L. C. Hsieh resolved SPARK-61556.
---------------------------------
    Fix Version/s: 4.4.0
       Resolution: Fixed

Issue resolved by pull request 60399
[https://github.com/apache/spark/pull/60399]

> Generate FilterExec predicate code with and without subexpression elimination 
> from one implementation
> -----------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61556
>                 URL: https://issues.apache.org/jira/browse/SPARK-61556
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 4.4.0
>            Reporter: L. C. Hsieh
>            Assignee: L. C. Hsieh
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.4.0
>
>
> {{FilterExec.doConsume}} re-implements, for the conjunctions it eliminates 
> common subexpressions from, the algorithm of 
> {{GeneratePredicateHelper.generatePredicateCode}} that orders the conjuncts 
> and the IsNotNull checks in the generated code. The two copies are kept in 
> sync only by comments ("Any change to this algorithm must be mirrored there 
> (and vice versa)"), and earlier fixes such as SPARK-51356 and SPARK-60315 had 
> to be applied to both.
> This is to have a single implementation: {{generatePredicateCode}} takes the 
> states of the common subexpressions, evaluates each right before the first 
> conjunct that references it, after the IsNotNull checks of that conjunct, and 
> generates the checks without the states. {{FilterExec.doConsume}} keeps the 
> decision to eliminate, the evaluation of the input variables and the creation 
> of the states.
> No behavior change is intended: the generated code without subexpression 
> elimination stays the same, and with it the code differs in whitespace only.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to