[
https://issues.apache.org/jira/browse/CALCITE-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027058#comment-17027058
]
Xiening Dai commented on CALCITE-3753:
--------------------------------------
{quote}
Roman Kondakov Calcite's engine always had the capability of "Cascades style
optimization with aggressive search space pruning". That is achieved by the
'importance' concept, the sorted queue of rule matches, and the ability to stop
optimization when the plan stops improving.
{quote}
I my opinion, space pruning and the current Calcite importance concept are
different. Space pruning is archived through top down optimization using
lower-bound, upper-bound calculation to eliminates alternatives that are
*guaranteed* to be worse. But Calcite rule importance setting is more heuristic
and cannot garantee the best plan is found. The "impatient" mode is
non-deterministic which makes it hardly useful in reality.
{quote}
Top-down is a subtlety in the Volcano paper that I missed. If top-down (or
something else) would solve the problem of requested traits then we should
consider it.
{quote}
I think Top-down is not just useful for requested traits, but also necessary
for space pruning - the lower-bound/upper-bound pruning can only be done
through top-down approach. Unfortunately current design of Calcite has many
aspects that would work against top-down searching. For example, in some cases,
an implementation rule (or even enforcement rule) can generate logical rel,
which then would require logic transformation to be applied again
(CALCITE-2970). So the plan might have to go back to the parent nodes again. If
we move to complete top down approach, we would have to put some limitations on
current RelOptRule (some interfaces change maybe), and then backward
compatibility would also become a problem.
> Always try to match and execute substitution rule first and remove rulematch
> ordering
> -------------------------------------------------------------------------------------
>
> Key: CALCITE-3753
> URL: https://issues.apache.org/jira/browse/CALCITE-3753
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Haisheng Yuan
> Priority: Major
> Attachments: image-2020-01-27-20-27-57-957.png
>
>
> In VolcanoPlanner, some rules e.g. ProjectMergeRule, PruneEmptyRule can be
> defined as SubstitutionRule, so that we can always try to match and execute
> them first (without deferring rule call). All the other rulematches doesn't
> need to be sorted and rules can be executed in any order they matched, since
> we are going to execute all of them anyway, sooner or later. Computing and
> comparing importances cause a lot of latency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
