[
https://issues.apache.org/jira/browse/CALCITE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081819#comment-17081819
]
Haisheng Yuan edited comment on CALCITE-3916 at 4/12/20, 2:39 PM:
------------------------------------------------------------------
Hi [~zabetak],
I understand that it might be a little bit confusing.
But before removing importance from subset, it was not top-down. The order is
controlled by importance, depending on the node cost and computing factor, it
can be any order. Even if the factor is properly tuned, rules are applied from
root RelSet to leaf RelSet, it is still impossible to apply space pruning. If
we have applied all the rules on top RelSet and we haven't applied rules on
child RelSet, how do we know what are the physical nodes of child RelSets, how
can we calculate the cost of child nodes, then how can we prune spaces?
Top-down optimizer doesn't mean the rules are applied top-down. It is
initiated, optimized, requested top-down. Here is how the process looks like,
and this is also what I want to achieve in this JIRA:
There are 3 stages for each RelSet:
1. Exploration: logical transformation, yields logical nodes
2. Implementation: physical transformation, yields physical nodes
3. Optimization: Physical property enforcement
Given a RelSet X, I want to optimize it, but before optimization, have it been
implemented? No, let's first implement this RelSet. Wait, have it been
explored? No, then let's first explore this RelSet. OK, how do we explore this
RelSet? Now, let's start from the first RelNode A in this RelSet to explore.
But, wait, have the child RelSets (Y and Z) of RelNode A been explored? No,
then let's first explore RelSet Y. and so on ...
Now X has been explored, let's implement this RelSet. Start from the first
logical RelNode A to implement. But have the child RelSets(Y and Z) been
implemented? No, let's first implement RelSet Y. ...
Basically it is a state machine with several state: Initialized, Explored,
Exploring, Implemented, Implementing, Optimized, Optimizing and several
transition methods: exploreRelSet, exploreRelNode, implementRelSet,
implementRelNode, optimizeRelSet, optimizeRelNode.
was (Author: hyuan):
Hi [~zabetak],
I understand that it might be a little bit confusing.
But before removing importance from subset, it was not top-down. The order is
controlled by importance, depending on the node cost and computing factor, it
can be any order. Even if the factor is properly tuned, rules are applied from
root RelSet to leaf RelSet, it is still impossible to apply space pruning. If
we have applied all the rules on top RelSet and we haven't applied rules on
child RelSet, how do we know what are the physical nodes of child RelSets, how
can we calculate the cost of child nodes, then how can we prune spaces?
Top-down optimizer doesn't mean the rules are applied top-down. It is
initiated, optimized, requested top-down. Here is how the process looks like,
and this is also what I want to achieve in this JIRA:
There are 3 stages for each RelSet:
1. Exploration: logical transformation, yields logical nodes
2. Implementation: physical transformation, yields physical nodes
3. Optimization: Physical property enforcement
Given a RelSet X, I want to optimize it, but before optimization, have it been
implemented? No, let's first implement this RelSet. Wait, have it been
explored? No, then let's first explore this RelSet. OK, how do we explore this
RelSet? Now, let's start from the first RelNode A in this RelSet to explore.
But, wait, have the child RelSets (Y and Z) of RelNode A been explored? No,
then let's first explore RelSet Y. and so on ...
Now X has been explored, let's implement this RelSet. Start from the first
logical RelNode A to implement. But have the child RelSets(Y and Z) been
implemented? No, let's first implement RelSet Y. ...
Basically it is a state machine with several state: Initialized, Explored,
Exploring, Implemented, Implementing, Optimized, Optimizing and several
transition methods: exploreRelSet, exploreRelNode, implementRelSet,
exploreRelNode, optimizeRelSet, optimizeRelNode.
> Apply rules bottom up by RelSet
> -------------------------------
>
> Key: CALCITE-3916
> URL: https://issues.apache.org/jira/browse/CALCITE-3916
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Haisheng Yuan
> Assignee: Haisheng Yuan
> Priority: Major
>
> Apply rules by leaf RelSet -> root RelSet order. For every RelNode in a
> RelSet, rule is matched and applied sequentially. No RuleQueue and
> DeferringRuleCall is needed anymore. This will make space pruning and rule
> mutual exclusivity check possible.
> Rule that use AbstractConverter as operand is an exception, to keep backward
> compatibility, this kind of rule still needs top-down apply.
> This should be done after CALCITE-3896.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
