[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944334#comment-16944334
]
Stamatis Zampetakis commented on CALCITE-2970:
--
I'm with you [~hyuan], that was exactly my point. I asked for a profiling just
to verify that there is not something obvious that we can do. Apart from that
another way to mitigate the problem would be to restrict the scope of rules
(especially for rules matching logical and physical operators) and/or organize
the rule application in phases.
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943729#comment-16943729
]
Haisheng Yuan commented on CALCITE-2970:
The analysis and solution looks valid at first glance, but it actually remove
the usage of abstract converter by directly adding sort operator for the inputs
of SortMergeJoin, thus it doesn't solve the issue fundamentally. Even if we
agree on the solution, what about the distribution? You have to create an
exchange operator on the inputs of MergeJoin, HashJoin and NestedLoopJoin, as
well as aggregate.
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943616#comment-16943616
]
Stamatis Zampetakis commented on CALCITE-2970:
--
Hey [~jinxing6...@126.com], thanks for working on this. I've seen that you did
some analysis to identify the cause of this issue. For the future, it is better
to include this kind of general information in the Jira rather than in the PR.
I think an important element that is missing from the analysis is the profiling
information. I do understand that the change in {{EnumerableMergeJoinRule}}
greatly improves the performance but how we can be sure that this rule is the
one (or the only one) that needs fixing. I suppose that you could equally
reduce the time by restricting matches in other rules.
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819232#comment-16819232
]
Chunwei Lei commented on CALCITE-2970:
--
{quote}Say we generate a sort merge join, if the child relation is not sorted
on the join key and abstract converter is not enabled, we won't get a sort
operator on the child relation, the consequence is: throwing an exception that
we can't plan it.
{quote}
Makes sense. Thank you for your explanation.
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819201#comment-16819201
]
Haisheng Yuan commented on CALCITE-2970:
Without enabling abstract converter (property enforcement), it won't be a
complete optimizer. Say we generate a sort merge join, if the child relation is
not sorted on the join key and abstract converter is not enabled, we won't get
a sort operator on the child relation, the consequence is: throwing an
exception that we can't plan it.
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816963#comment-16816963
]
Chunwei Lei commented on CALCITE-2970:
--
It seems {{checkJoinNWay(6)}} will not complete if enabling the use of abstract
converter for {{EnumerableConvention.}}
Why do you want to enable it?
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
