[
https://issues.apache.org/jira/browse/CALCITE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013410#comment-17013410
]
Vladimir Sitnikov commented on CALCITE-2356:
--------------------------------------------
[~danny0405], I agree there's something to be desired there with rule execution
order, trait propagation.
I just wanted to try a small change first, because the current implementation
suffers from rule starvation (infinite match of a single rule).
Does PR#1732 work? Can we try running slow tests with it?
My naive ordering implementation in
[https://github.com/apache/calcite/pull/1745] improves slow tests to 22 minutes
(current master takes ~37m).
The improvement to FoodmartTest is significant: only ~30 out of 6310 queries
exceed 2 seconds
[https://github.com/apache/calcite/pull/1745/checks?check_run_id=384236670#step:4:446]
{noformat}
946.1sec, 6310 completed, 0 failed, 0 skipped,
org.apache.calcite.test.FoodmartTest
{noformat}
master (110 queries exceed 2sec):
[https://github.com/apache/calcite/commit/e96a9aaec6cb75be65cbe509d3001b2533a29706/checks?check_suite_id=394473874#step:4:493]
{noformat}
1349.5sec, 6310 completed, 0 failed, 0 skipped,
org.apache.calcite.test.FoodmartTest
{noformat}
I have added a PR that executes the rules in random order, and its performance
is comparable to the master. It might be worth considering as a baseline.
[https://github.com/apache/calcite/pull/1747/checks?check_run_id=384628103#step:4:537]
{noformat}
there's StackOverflowEror in one of Foodmart queries
FAILURE 1301.8sec, 6310 completed, 1 failed, 0 skipped,
org.apache.calcite.test.FoodmartTest
{noformat}
> Allow user defined policy to dynamically define all or some specific rules'
> execution order or even skip some rules
> -------------------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-2356
> URL: https://issues.apache.org/jira/browse/CALCITE-2356
> Project: Calcite
> Issue Type: Bug
> Reporter: Chunhui Shi
> Priority: Major
>
> We have seen the order of the rule execution did impact VolcanoPlanner's
> behavior, for example, in CALCITE-2223, if we reverse the order decided by
> name in RuleMatchImportanceComparator, we could get different result for
> CALCITE-2223 case.
> And in some of our practices, we have seen rules on overlapped patterns could
> also trigger unnecessary chaos and much bigger exploration space which caused
> the planning time became much longer.
> So the proposal of this Jira is, while each rule focuses on the local
> pattern, Calcite allow a pluggable coordinator of rule execution to utilize
> the knowledge we have about the rules and current state. The output of this
> coordinator is the sequence of rules to execute on matching patterns. The
> input is still the matching rules and pattern discovered by Calcite. When new
> nodes added and new rules need to be triggered, they should be added through
> the coordinator to decide whether/how they will be executed.
> This proposed feature should not impact any current Calcite users since they
> don't define their own policies.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
