[
https://issues.apache.org/jira/browse/SPARK-61556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L. C. Hsieh reassigned SPARK-61556:
-----------------------------------
Assignee: L. C. Hsieh
> Generate FilterExec predicate code with and without subexpression elimination
> from one implementation
> -----------------------------------------------------------------------------------------------------
>
> Key: SPARK-61556
> URL: https://issues.apache.org/jira/browse/SPARK-61556
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: L. C. Hsieh
> Assignee: L. C. Hsieh
> Priority: Major
> Labels: pull-request-available
>
> {{FilterExec.doConsume}} re-implements, for the conjunctions it eliminates
> common subexpressions from, the algorithm of
> {{GeneratePredicateHelper.generatePredicateCode}} that orders the conjuncts
> and the IsNotNull checks in the generated code. The two copies are kept in
> sync only by comments ("Any change to this algorithm must be mirrored there
> (and vice versa)"), and earlier fixes such as SPARK-51356 and SPARK-60315 had
> to be applied to both.
> This is to have a single implementation: {{generatePredicateCode}} takes the
> states of the common subexpressions, evaluates each right before the first
> conjunct that references it, after the IsNotNull checks of that conjunct, and
> generates the checks without the states. {{FilterExec.doConsume}} keeps the
> decision to eliminate, the evaluation of the input variables and the creation
> of the states.
> No behavior change is intended: the generated code without subexpression
> elimination stays the same, and with it the code differs in whitespace only.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]