[
https://issues.apache.org/jira/browse/BEAM-4388?focusedWorklogId=107211&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-107211
]
ASF GitHub Bot logged work on BEAM-4388:
----------------------------------------
Author: ASF GitHub Bot
Created on: 30/May/18 15:45
Start Date: 30/May/18 15:45
Worklog Time Spent: 10m
Work Description: kennknowles commented on a change in pull request
#5481: [BEAM-4388] Support optimized logical plan
URL: https://github.com/apache/beam/pull/5481#discussion_r191815818
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/planner/BeamRuleSets.java
##########
@@ -39,22 +60,91 @@
* RelNode} tree, to represent with {@link BeamRelNode}
*/
public class BeamRuleSets {
+ public static final RuleSet LOGICAL_OPT_RULES =
+ RuleSets.ofList(
+ // translate to beam logical rel nodes
+ BeamAggregationRule.INSTANCE,
+ BeamEnumerableConverterRule.INSTANCE,
+ BeamFilterRule.INSTANCE,
+ BeamIntersectRule.INSTANCE,
+ BeamIOSinkRule.INSTANCE,
+ BeamJoinRule.INSTANCE,
+ BeamMinusRule.INSTANCE,
+ BeamProjectRule.INSTANCE,
+ BeamSortRule.INSTANCE,
+ BeamUnionRule.INSTANCE,
+ BeamUncollectRule.INSTANCE,
+ BeamUnnestRule.INSTANCE,
+ BeamValuesRule.INSTANCE,
+
+ // push a filter into a join
+ FilterJoinRule.FILTER_ON_JOIN,
+ // push filter into the children of a join
+ FilterJoinRule.JOIN,
+ // push filter through an aggregation
+ FilterAggregateTransposeRule.INSTANCE,
+ // push filter through set operation
+ FilterSetOpTransposeRule.INSTANCE,
+ // push project through set operation
+ ProjectSetOpTransposeRule.INSTANCE,
+
+ // push a projection past a filter or vice versa
+ ProjectFilterTransposeRule.INSTANCE,
+ FilterProjectTransposeRule.INSTANCE,
+
+ // push a projection to the children of a join
+ // push all expressions to handle the time indicator correctly
+ new ProjectJoinTransposeRule(
+ PushProjector.ExprCondition.FALSE, RelFactories.LOGICAL_BUILDER),
+ // merge projections
+ ProjectMergeRule.INSTANCE,
+ // reorder sort and projection
+ SortProjectTransposeRule.INSTANCE,
+ ProjectSortTransposeRule.INSTANCE,
+
+ // join rules
+ JoinPushExpressionsRule.INSTANCE,
+
+ // remove union with only a single child
+ UnionEliminatorRule.INSTANCE,
+ // convert non-all union into all-union + distinct
+ UnionToDistinctRule.INSTANCE,
+
+ // remove aggregation if it does not aggregate and input is already
distinct
+ AggregateRemoveRule.INSTANCE,
+ // push aggregate through join
+ AggregateJoinTransposeRule.EXTENDED,
+ // aggregate union rule
+ AggregateUnionAggregateRule.INSTANCE,
+
+ // remove unnecessary sort rule
+ SortRemoveRule.INSTANCE,
+
+ // prune empty results rules
+ PruneEmptyRules.AGGREGATE_INSTANCE,
+ PruneEmptyRules.FILTER_INSTANCE,
+ PruneEmptyRules.JOIN_LEFT_INSTANCE,
+ PruneEmptyRules.JOIN_RIGHT_INSTANCE,
+ PruneEmptyRules.PROJECT_INSTANCE,
+ PruneEmptyRules.SORT_INSTANCE,
+ PruneEmptyRules.UNION_INSTANCE);
public static RuleSet[] getRuleSets() {
return new RuleSet[] {
RuleSets.ofList(
- BeamProjectRule.INSTANCE,
- BeamFilterRule.INSTANCE,
BeamAggregationRule.INSTANCE,
- BeamSortRule.INSTANCE,
- BeamValuesRule.INSTANCE,
+ BeamEnumerableConverterRule.INSTANCE,
Review comment:
Probably this method can just return `List<RuleSet>` and we can have
separate rule sets, just for organization. Because actually the way the
`RuleSet[]` is passed in `BeamQueryPlanner` is sort of abusing varargs.
I don't really know what separating it into multiple rulesets does except
for being able to pass them around separately easily.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 107211)
> Support optimized logical plan
> ------------------------------
>
> Key: BEAM-4388
> URL: https://issues.apache.org/jira/browse/BEAM-4388
> Project: Beam
> Issue Type: Sub-task
> Components: dsl-sql
> Reporter: Kai Jiang
> Assignee: Kai Jiang
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Before converting into Beam Pipeline physical plan, logical plan should be
> optimized and it will be super helpful for efficiently executing Beam
> PTransforms pipeline.
> Calcite has two ways for optimizing logical plan (HepPlanner and
> VolcanoPlanner). We can support VolcanoPlanner first and apply calcite
> builtin optimize rules (like
> FilterJoinRule.FILTER_ON_JOIN) to sql query optimize plans.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
