[
https://issues.apache.org/jira/browse/CALCITE-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120446#comment-17120446
]
Anton Haidai commented on CALCITE-4029:
---------------------------------------
This is a continuation of discussion in
https://issues.apache.org/jira/browse/CALCITE-3939 comments.
[~botong],
{quote}
[~anha] Understood (we are on the same page for Step 2). But still I don't
think this is a bug.
1. Both before and after the RelSet merge, if there's any other logical relNode
in the same RelSet as the trivial project. It will get a match and the RuleX
will fire on it.
2. Logical project/sort having a non logical CustomScan input is a weird
design. This is the fundamental reason you are seeing this issue. If you really
want it to fire, consider changing the child operand in RuleX to match any
convention. (btw, when you write operand(LogicalSort.class,
operand(RelNode.class, any())) earlier, this should already match the
customScan with a custom convention right?)
{quote}
>From you comment I see that you don't have the full understanding of the
>trick. <TrivialProject> discarded on Step 2 is only a part of the problem.
LogicalSort input is a RelSubset (not a RelSet!) in the same NONE convention
(RelSubset#19 or rel#19 in the example below) and the rule fires only when this
RelSubset#19 is non-empty because it expects LogicalSort with it's input node
(in NONE convention, even if RuleX does not explicitly request it in it's
operand(...)).
Let's compare simplified sets dumps.
*autoPruneOld() = true:*
{code:java}
Sets:
Set#3
rel#10:Subset#0.FINAL_CONVENTION
CustomScan.FINAL_CONVENTION
rel#19:Subset#0.NONE.[].NOT_QUERY.CUSTOM_RULE_NON_PROCESSABLE
Set#1
rel#14:Subset#2.NONE.[].NOT_QUERY.CUSTOM_RULE_NON_PROCESSABLE, best=null
LogicalSort.NONE.[](input=RelSubset#19,fetch=5)){code}
*"LogicalSort*.NONE.[](*input*=*RelSubset#19*,fetch=5))".
Here, RelSubset#19 is empty (which is a LogicalSort's input). So RuleX can't
fire: while "operand(RelNode.class, any()))" does not specify the convention of
a input node, it still requires this input node, but the input of a LogicalSort
is empty.
*autoPruneOld() = false:*
{code:java}
Sets:
Set#3
rel#10:Subset#0.FINAL_CONVENTION
CustomScan.FINAL_CONVENTION
rel#19:Subset#0.NONE.[].NOT_QUERY.CUSTOM_RULE_NON_PROCESSABLE
LogicalProject.NONE
Set#1
rel#14:Subset#2.NONE.[].NOT_QUERY.CUSTOM_RULE_NON_PROCESSABLE, best=null
LogicalSort.NONE.[](input=RelSubset#19,fetch=5)){code}
Here, RelSubset#19 is not empty (it still has the TrivialProject) so RuleX can
fire.
{quote}
If you really want it to fire, consider changing the child operand in RuleX to
match any convention. (btw, when you write operand(LogicalSort.class,
operand(RelNode.class, any())) earlier, this should already match the
customScan with a custom convention right?
{quote}
So the answer is "no" as explained above.
{quote}
Logical project/sort having a non logical CustomScan input is a weird design.
This is the fundamental reason you are seeing this issue.
{quote}
AFAIR I read somewhere that having leaf node in a final convention is
beneficial for planning, but I can't find the source and can't figure out a
motivation so possibly it is not correct.
Using LogicalScan - > CustomScanRule - > CustomScan will help.
As a toy PoC, I wrote a simple rule that matches CustomScan and transforms it
into a dummy unimplementable Logical node, and this also fixes the issue (this
dummy node is added into RelSubset#19 enabling RuleX to fire).
I'm also not sure if this ticket is a bug or not, so if it doesn't look like a
bug for Calcite's developers feel free to just close it.
> ProjectRemoveRule auto pruning may prevent rules from running if mixed
> conventions are used in a logical plan
> --------------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-4029
> URL: https://issues.apache.org/jira/browse/CALCITE-4029
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.23.0
> Reporter: Anton Haidai
> Priority: Minor
>
> Preconditions to reproduce the issue:
> # Logical plan has mixed conventions (for example, a bottom node is a
> TableScan in a final convention while other nodes are regular logical nodes
> with NONE convention).
> # There is a rule that expects a logical node with an input (like a rule
> matching "operand(LogicalSort.class, operand(RelNode.class, any()))")
> # A project over the scan is trivial (like SELECT * FROM ...)
> The issue is related to https://issues.apache.org/jira/browse/CALCITE-3939,
> please see comments for a detailed debugging of a real-life reproducing case.
> h4. Example:
> Logical plan with a leaf nodes in a custom convention:
> {code:java}
> LogicalSort[NONE]
> LogicalProject[NONE]
> CustomScan[CUSTOM_CONVENTION]{code}
> A rule configured (RuleX) matches "operand(LogicalSort.class,
> operand(RelNode.class, any()))".
> *Without ProjectRemoveRule auto pruning*
> ProjectRemoveRule recognizes LogicalProject as trivial an merges it into a
> single RelSet with CustomScan.
> RuleX can run on top of this change as far as LogicalProject has a logical
> node (LogicalProject in RelSubset[NONE]) as an input.
>
> *With ProjectRemoveRule auto pruning*
> ProjectRemoveRule recognizes LogicalProject as trivial but removes it with
> it's RelSet so the CustomScan is the only node in it's RelSet,
> RelSubset[CUSTOM_CONVENTION].
> RuleX can't run on top of this change as far as LogicalProject has an empty
> input RelSubset[NONE] of the RelSet with the CustomScan.
> h2. Possible workarounds
> # Disable ProjectRemoveRule auto pruning.
> # Use only logical nodes in a logical plan, for the example above: use
> LogicalScan - > CustomScanRule - > CustomScan instead of direct use of
> CustomScan.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
