[
https://issues.apache.org/jira/browse/CALCITE-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119048#comment-17119048
]
Anton Haidai commented on CALCITE-3939:
---------------------------------------
[~botong], thank you for you detailed description. I did some debugging and I
think that I figured out what is going on, so here are debugging results
aligned with the steps from your message.
h2. Step 1
<RuleMatch1> happens as described.
h2. Step 2
Due to the autoPruneOld() returning true, there is a call
volcanoPlanner.prune(<TrivialProject>) before the merge of <RelSet2> and
<RelSet3>.
The merge itself: there is a call <RelSet3>.mergeWith(<RelSet2>)
So the <RelSet2> is being removed from a list of sets.
There is an attempt to add <TrivialProject> into <RelSet3>:
planner.reregister(<RelSet3>, <TrivialProject>);
But this VolcanoPlanner.reregister() method has the following check:
{code:java}
if (!prunedNodes.contains(rel)) {
addRelToSet(rel, set);
}{code}
So there is a difference with your scenario: instead of both <TrivialProject>
and <CustomScan> being in <RelSet3>, there is only <CustomScan> there while the
<TrivialProject> is just thrown away.
h2. Step 3
This <RuleMatch2> never happens. I think, this is because <LogicalSort>
obviously has a convention NONE as a logical node, but <CustomScan> is *not* a
logical node and has some final custom convention from the beginning. So while
the <CustomScan> is the only node in a set, there is no other nodes with NONE
convention in this set so <LogicalSort> input is an empty subset with NONE
convention and <RuleX> can't match once more and <RuleMatch2> never happens.
h2. Step 4
all as described, <RuleMatch1> is ignored (RuleQueue.skipMatch() method).
h2. Step 5
As described in Step 3 <RuleMatch2> never happens.
My workaround (force ProjectRemoveRule.autoPruneOld() to return false) make it
work exactly as you described: <RelSet3> contains both <CustomScan> and
<TrivialProject> making <RuleMatch2> possible because <LogicalSort> has some
input with NONE convention.
Another workaround could be using something like <LogicalScan>[NONE] ->
<CustomScanRule> -> <CustomScan>[FINAL_CONVENTION] instead of a direct use of
<CustomScan>[FINAL_CONVENTION] in a logical plan: pretty sure it will work, but
I didn't check it.
> Change UnionEliminatorRule and ProjectRemoveRule to auto pruning
> SubstitutionRule
> ---------------------------------------------------------------------------------
>
> Key: CALCITE-3939
> URL: https://issues.apache.org/jira/browse/CALCITE-3939
> Project: Calcite
> Issue Type: Improvement
> Reporter: Botong Huang
> Priority: Major
> Fix For: 1.23.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> UnionEliminatorRule and ProjectRemoveRule are both pruning rules for a
> RelNode. They can also become SubstitutionRule with autoprune enabled
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
