[
https://issues.apache.org/jira/browse/BEAM-10595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171060#comment-17171060
]
Rui Wang commented on BEAM-10595:
---------------------------------
Also this change does not affect performance:
Our rule set before the change contains:
ProjectCalcMergeRule
FilterCalcMergeRule
ProjectToCalcRule
FilterToCalcRule
CalcMergeRule
BeamCalcRule (BeamZetaSqlCalcRule)
As you can see, there are two paths to eventually generate our single
BeamCalcRel(BeamZetaSqlCalcRel):
{code:java}
Path one: Projects/Filters ----------------> Calcs ------------------->
all Calc merged-------------------------------------------------------> final
Calc Rel
|
| |
ProjectToCalcRule CalcMergeRule
BeamCalcRule (BeamZetaSqlCalcRule)
FilterToCalcRule
Path two: Projects/Filters ----------------> Calcs ---------------------->
Calcs merge with not-converted Projects/Filters --------------> final Calc Rel
|
|
|
ProjectToCalcRule
ProjectCalcMergeRule
BeamCalcRule (BeamZetaSqlCalcRule)
FilterToCalcRule
FilterCalcMergeRule
{code}
So after removing ProjectCalcMergeRule and FilterCalcMergeRule, it will reduce
search space during planning, thus it is not a performance regression. In fact,
it is a performance improvement (although the improvement is minor compared to
the big data processing workload).
> Remove rules that not work well with Java UDF
> ---------------------------------------------
>
> Key: BEAM-10595
> URL: https://issues.apache.org/jira/browse/BEAM-10595
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql-zetasql
> Reporter: Rui Wang
> Assignee: Rui Wang
> Priority: P2
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> In order to support Java UDF, we need both BeamZetaSqlCalcRel and
> BeamCalcRel. It is
> because BeamZetaSqlCalcRel can execute ZetaSQL built-in functions while
> BeamCalcRel
> can execute UDFs. So during planning, we expect both Filter and Project are
> converted to
> a Calc before merging with other Projec/Filter/Calc. Thus we should not add
> FilterCalcMergeRuleand ProjectCalcMergeRule.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
