[
https://issues.apache.org/jira/browse/CALCITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013055#comment-17013055
]
Vladimir Sitnikov commented on CALCITE-2356:
--------------------------------------------
Here's a detailed step-by-step how CALCITE-2223 breaks:
https://issues.apache.org/jira/browse/CALCITE-2223?focusedCommentId=16735127&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16735127
TL;DR: it produces lots of relations of the following kind:
{noformat}LogicalUnion(all=[true])
LogicalFilter(subset=[rel#363:Subset#23.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#352:Subset#21.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#317:Subset#19.NONE.[]], condition=[>($0, 10)])
LogicalProject(subset=[rel#312:Subset#18.NONE.[]], deptno=[$0])
LogicalFilter(subset=[rel#310:Subset#17.NONE.[]], condition=[>=(30,
$0)])
LogicalJoin(subset=[rel#216:Subset#4.NONE.[]], condition=[=($1,
$2)],
{noformat}
In practice, it fails to recognize that extra filter adds nothing.
FilterMergeRule is there, however it is not selected for execution due to
RuleMatchImportanceComparator which compares rules by Class names or something
like that.
What if we classify the rules as "the rule that expands planning space" vs "the
rule that shrinks planning space"?
Then we could prefer picking the rules that shrink planning space.
The classification can be either manual (e.g. a method override in the rule
definition) or dynamic. For instance, the planner could count the number of new
relations that was produced by a rule, then it could prefer selecting the rules
that shrink the planning space (e.g. result in merges of subsets).
By expanding the planning space I mean the number of generated subsets and the
number of newly added relations to the subsets.
WDYT?
> Allow user defined policy to dynamically define all or some specific rules'
> execution order or even skip some rules
> -------------------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-2356
> URL: https://issues.apache.org/jira/browse/CALCITE-2356
> Project: Calcite
> Issue Type: Bug
> Reporter: Chunhui Shi
> Priority: Major
>
> We have seen the order of the rule execution did impact VolcanoPlanner's
> behavior, for example, in CALCITE-2223, if we reverse the order decided by
> name in RuleMatchImportanceComparator, we could get different result for
> CALCITE-2223 case.
> And in some of our practices, we have seen rules on overlapped patterns could
> also trigger unnecessary chaos and much bigger exploration space which caused
> the planning time became much longer.
> So the proposal of this Jira is, while each rule focuses on the local
> pattern, Calcite allow a pluggable coordinator of rule execution to utilize
> the knowledge we have about the rules and current state. The output of this
> coordinator is the sequence of rules to execute on matching patterns. The
> input is still the matching rules and pattern discovered by Calcite. When new
> nodes added and new rules need to be triggered, they should be added through
> the coordinator to decide whether/how they will be executed.
> This proposed feature should not impact any current Calcite users since they
> don't define their own policies.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
