[jira] [Comment Edited] (CALCITE-2018) Queries failed with AssertionError: rel has lower cost than best cost of subset
[
https://issues.apache.org/jira/browse/CALCITE-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962670#comment-16962670
]
Danny Chen edited comment on CALCITE-2018 at 10/30/19 4:24 AM:
---
Fixed in
[e0d869a|https://github.com/apache/calcite/commit/e0d869ab8e540dec71b9278f5a3202df4a2dfb5c],
[d3b1d85|https://github.com/apache/calcite/commit/d3b1d857380972b7ca7fa5b70f9903977054594c],
[ad90589|https://github.com/apache/calcite/commit/ad9058999bdea1329dba2267604512d8e199919f],
thanks for the great work [~volodymyr] and [~xndai] !
And thanks all the colleagues participate in pushing this forward !
was (Author: danny0405):
Fixed in
[e0d869a|https://github.com/apache/calcite/commit/e0d869ab8e540dec71b9278f5a3202df4a2dfb5c],
[d3b1d85|https://github.com/apache/calcite/commit/d3b1d857380972b7ca7fa5b70f9903977054594c],
[0312265|https://github.com/apache/calcite/commit/03122652d1ad50fb900cab003bf18433a07cdf5b],
thanks for the great work [~volodymyr] and [~xndai] !
And thanks all the colleagues participate in pushing this forward !
> Queries failed with AssertionError: rel has lower cost than best cost of
> subset
> ---
>
> Key: CALCITE-2018
> URL: https://issues.apache.org/jira/browse/CALCITE-2018
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Vova Vysotskyi
>Assignee: Danny Chen
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> *Problem description*
> When rootLogger level is DEBUG, unit tests
> * MaterializationTest.testMaterializationSubstitution2
> * MaterializationTest.testJoinMaterializationUKFK8
> * MaterializationTest.testJoinMaterializationUKFK6
> * JdbcTest.testWhereNot
> unit tests are failed with error AssertionError: rel has lower cost than best
> cost of subset.
> Full stack trace for test
> {{MaterializationTest.testMaterializationSubstitution2}}:
> {noformat}
> java.lang.AssertionError: rel
> [rel#245:EnumerableUnion.ENUMERABLE.[](input#0=rel#246:Subset#5.ENUMERABLE.[],input#1=rel#239:Subset#6.ENUMERABLE.[0],all=true)]
> has lower cost {14.0 rows, 19.0 cpu, 0.0 io} than best cost {15.0 rows, 20.0
> cpu, 0.0 io} of subset [rel#243:Subset#7.ENUMERABLE.[]]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.validate(VolcanoPlanner.java:906)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:866)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:101)
> at
> org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:336)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1495)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:863)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1766)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135)
> at
> org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
> at
> org.apache.calcite.rel.rules.FilterProjectTransposeRule.onMatch(FilterProjectTransposeRule.java:143)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650)
> at org.apache.calcite.tools.Programs$5.run(Programs.java:326)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:187)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:318)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
> at org.apache.calcite.schema.Schemas.prepare(Schemas.java:346)
> at
> org.apache.calcite.materialize.MaterializationService$DefaultTableFactory.createTable(MaterializationService.java:374)
> at
> org.apache.calcite.materialize.MaterializationService.defineMaterialization(MaterializationService.java:137)
> at
> org.apache.calcite.materialize.M
[jira] [Comment Edited] (CALCITE-2018) Queries failed with AssertionError: rel has lower cost than best cost of subset
[
https://issues.apache.org/jira/browse/CALCITE-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944646#comment-16944646
]
Xiening Dai edited comment on CALCITE-2018 at 10/4/19 4:28 PM:
---
The cyclic metadata exception is due to the change of memo that's caused by the
fix of CALCITE-2018. Actually this is a separate issue and we saw it in
different queries even before the fix. I don't think it is caused by the
validation. In the validation step, I choose to create a new RelMetadataQuery
object each time so it doesn't affect the cache of volcano planner in any way.
The only side effect I can see is the test running time could be extended due
to the extra validation steps (which is not cheap). But I think it worths it as
it catches any correctness and consistency issues of memo right the way.
was (Author: xndai):
The cyclic metadata exception is due to the change of memo that's caused by the
fix of CALCITE-2018. Actually this is a separate issue and we saw it in
different queries even before the fix. I don't think it relates to the
validation. In the validation step, I choose to create a new RelMetadataQuery
object each time so it doesn't affect the cache of volcano planner in any way.
The only side effect I can see is the test running time could be extended due
to the extra validation steps (which is not cheap). But I think it worths it as
it catches any correctness and consistency issues of memo right the way.
> Queries failed with AssertionError: rel has lower cost than best cost of
> subset
> ---
>
> Key: CALCITE-2018
> URL: https://issues.apache.org/jira/browse/CALCITE-2018
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Vova Vysotskyi
>Assignee: Danny Chen
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> *Problem description*
> When rootLogger level is DEBUG, unit tests
> * MaterializationTest.testMaterializationSubstitution2
> * MaterializationTest.testJoinMaterializationUKFK8
> * MaterializationTest.testJoinMaterializationUKFK6
> * JdbcTest.testWhereNot
> unit tests are failed with error AssertionError: rel has lower cost than best
> cost of subset.
> Full stack trace for test
> {{MaterializationTest.testMaterializationSubstitution2}}:
> {noformat}
> java.lang.AssertionError: rel
> [rel#245:EnumerableUnion.ENUMERABLE.[](input#0=rel#246:Subset#5.ENUMERABLE.[],input#1=rel#239:Subset#6.ENUMERABLE.[0],all=true)]
> has lower cost {14.0 rows, 19.0 cpu, 0.0 io} than best cost {15.0 rows, 20.0
> cpu, 0.0 io} of subset [rel#243:Subset#7.ENUMERABLE.[]]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.validate(VolcanoPlanner.java:906)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:866)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:101)
> at
> org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:336)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1495)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:863)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1766)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135)
> at
> org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
> at
> org.apache.calcite.rel.rules.FilterProjectTransposeRule.onMatch(FilterProjectTransposeRule.java:143)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650)
> at org.apache.calcite.tools.Programs$5.run(Programs.java:326)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:187)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:318)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
>
[jira] [Comment Edited] (CALCITE-2018) Queries failed with AssertionError: rel has lower cost than best cost of subset
[
https://issues.apache.org/jira/browse/CALCITE-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350711#comment-16350711
]
Volodymyr Vysotskyi edited comment on CALCITE-2018 at 2/2/18 5:41 PM:
--
[~julianhyde], I tried to add these checks for cached and uncached metadata
values, and all tests in Calcite are passed. But when I ran Drill unit tests
with this check, some of them failed.
This failure appeared because *the cumulative cost of best rel node increased*.
For example when we have such piece of the plan:
{noformat}
SingleMergeExchangePrel(sort0=[1 DESC]): rowcount = 60.175, cumulative cost =
{60.175 rows, 481.4 cpu, 0.0 io, 492953.6 network, 0.0 memory}, id = 2232
SortPrel(subset=[rel#2228:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$1]]).[1
DESC]], sort0=[$1], dir0=[DESC]): rowcount = 60.175, cumulative cost = {60.175
rows, 1422.7999242130231 cpu, 0.0 io, 0.0 network, 962.8 memory}, id = 2896
HashToRandomExchangePrel(subset=[rel#2885:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$1]]).[]],
dist0=[[$1]]): rowcount = 60.175, cumulative cost = {60.175 rows, 962.8 cpu,
0.0 io, 492953.6 network, 0.0 memory}, id = 2918
StreamAggPrel(subset=[rel#2900:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$0]]).[]],
group=[{0}], revenue=[SUM($1)]): rowcount = 60.175, cumulative cost = {601.75
rows, 9628.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2215
HashToMergeExchangePrel(subset=[rel#2214:Subset#204.PHYSICAL.HASH_DISTRIBUTED([[$0]]).[0]]):
rowcount = 601.75, cumulative cost = {601.75 rows, 9628.0 cpu, 0.0 io,
4929536.0 network, 0.0 memory}, id = 2213
StreamAggPrel(subset=[rel#2205:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$0]]).[0]],
group=[{0}], revenue=[SUM($1)]): rowcount = 601.75, cumulative cost = {6017.5
rows, 96280.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2472
{noformat}
rel node {{SingleMergeExchangePrel}} has a smaller cumulative cost than the
cumulative cost of the same rel node after applying some rules:
{noformat}
SingleMergeExchangePrel(sort0=[1 DESC]): rowcount = 601.75, cumulative cost =
{601.75 rows, 4814.0 cpu, 0.0 io, 4929536.0 network, 0.0 memory}, id = 2232
SortPrel(subset=[rel#2228:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$1]]).[1
DESC]], sort0=[$1], dir0=[DESC]): rowcount = 601.75, cumulative cost = {601.75
rows, 3.88016652411 cpu, 0.0 io, 0.0 network, 9628.0 memory}, id = 2896
HashToRandomExchangePrel(subset=[rel#2885:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$1]]).[]],
dist0=[[$1]]): rowcount = 601.75, cumulative cost = {601.75 rows, 9628.0 cpu,
0.0 io, 4929536.0 network, 0.0 memory}, id = 2884
StreamAggPrel(subset=[rel#2205:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$0]]).[0]],
group=[{0}], revenue=[SUM($1)]): rowcount = 601.75, cumulative cost = {6017.5
rows, 96280.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2472
{noformat}
But its cost was changed because the cumulative cost of {{SortPrel}} from the
second plan is less than the cost of the same rel node from the first plan.
Perhaps we should change a little bit cost estimation in Drill, but I am not
sure that it will help to prevent other similar cases.
Or we should try to modify Volcano planner to be able to revert the change of
best rel nodes when the cumulative cost of neither of top rel nodes wasn't
decreased.
Or we may reflect the increase in the cumulative cost of best rel node in
bestCost.
[~julianhyde], [~amansinha100], could you please advise me the better way to
fix this issue?
was (Author: vvysotskyi):
[~julianhyde], I tried to add these checks for cached and uncached metadata
values, and all tests in Calcite are passed. But when I ran Drill unit tests
with this check, some of them failed.
This failure appeared because *the cost of best rel node increased*. For
example when we have such piece of the plan:
{noformat}
SingleMergeExchangePrel(sort0=[1 DESC]): rowcount = 60.175, cumulative cost =
{60.175 rows, 481.4 cpu, 0.0 io, 492953.6 network, 0.0 memory}, id = 2232
SortPrel(subset=[rel#2228:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$1]]).[1
DESC]], sort0=[$1], dir0=[DESC]): rowcount = 60.175, cumulative cost = {60.175
rows, 1422.7999242130231 cpu, 0.0 io, 0.0 network, 962.8 memory}, id = 2896
HashToRandomExchangePrel(subset=[rel#2885:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$1]]).[]],
dist0=[[$1]]): rowcount = 60.175, cumulative cost = {60.175 rows, 962.8 cpu,
0.0 io, 492953.6 network, 0.0 memory}, id = 2918
StreamAggPrel(subset=[rel#2900:Subset#201.PHYSICAL.HASH_DISTRIBUTED([[$0]]).[]],
group=[{0}], revenue=[SUM($1)]): rowcount = 60.175, cumulative cost = {601.75
rows, 9628.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 2215
HashToMergeExchangePrel(subset=[rel#2214:Subset#204.PHYSICAL.HASH_DISTRIBUTED([[$0]]).[0]]):
rowcount = 601.75, cumulative cost = {601.75 rows, 9628.0 cpu, 0.0 io,
4929536.0 network, 0.0 me
[jira] [Comment Edited] (CALCITE-2018) Queries failed with AssertionError: rel has lower cost than best cost of subset
[
https://issues.apache.org/jira/browse/CALCITE-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300355#comment-16300355
]
Volodymyr Vysotskyi edited comment on CALCITE-2018 at 12/28/17 1:01 PM:
[~jcamachorodriguez], considering rels in RelSubset, I observed cases where
such metadata as row count is different for few rels from the same RelSet. So I
don't think that we can be sure that some kind of metadata left unchanged.
Currently, I am reworking this fix.
Main points that already done:
0) Left the same metadata cache structure as it was in my original PR. It
allows finding and deleting all cached values for the same RelNode easier.
1) Made changes to use the same {{RelOptCluster}} instance for RelNodes, so it
makes possible to use single {{RelMetadataQuery}} instance between the first
call of {{RelOptCluster.getMetadataQuery()}} and
{{RelOptCluster.invalidateMetadataQuery()}} call.
2) Cached metadata values are removed for current {{RelSubset}} and all parent
{{RelNode}} s when {{RelSubset.propagateCostImprovements()}} is called and best
rel node was changed.
3) Cached metadata values are removed for current {{RelNode}} when its input
was changed.
4) Made changes in {{RelSet.mergeWith()}} method to call
{{RelSubset.propagateCostImprovements()}} when resulting best is known instead
of just assigning {{RelSubset.best}}. It is needed for the cases when
{{getOrCreateSubset()}} method returns existing {{RelSubset}}, so we need to
inform its parent rels that its best rel node was changed. (Currently, only
otherSubset parents are informed.)
was (Author: vvysotskyi):
[~jcamachorodriguez], considering rels in RelSubset, I observed cases where
such metadata as row count is different for few rels from the same RelSet. So I
don't think that we can be sure that some kind of metadata left unchanged.
Currently, I am reworking this fix.
Main points that already done:
0) Left the same metadata cache structure as it was in my original PR. It
allows finding and deleting all cached values for the same RelNode easier.
1) Made changes to use the same {{RelOptCluster}} instance for RelNodes, so it
makes possible to use single {{RelMetadataQuery}} instance between the first
call of {{RelOptCluster.getMetadataQuery()}} and
{{RelOptCluster.invalidateMetadataQuery()}} call.
2) Cached metadata values are removed for current {{RelSubset}} and all parent
{{RelNode}} s when {{RelSubset.propagateCostImprovements()}} is called and best
rel node was changed.
3) Cached metadata values are removed for current {{RelNode}} when its input
was changed.
4) Made changes in {{RelSet.mergeWith()}} method to call
{{RelSubset.propagateCostImprovements()}} when resulting best is known instead
of just assigning {{RelSubset.best}}. It is needed for the cases when
{{getOrCreateSubset()}} method returns existing {{RelSubset}}, so we need to
inform its parent rels that its best rel node was changed. (Currently, only
otherSubset parents are informed.)
Currently, left single unresolved issue:
When {{RelNode}} instance is passed into
{{RelSubset.propagateCostImprovements()}} method, its parent subset is taken
from the {{VolcanoPlanner.mapRel2Subset}} map. If exists another subset, whose
{{RelTraitSet}} is satisfied by {{RelNode}} instance {{RelTraitSet}},
{{VolcanoPlanner.validate()}} method throws an error.
> Queries failed with AssertionError: rel has lower cost than best cost of
> subset
> ---
>
> Key: CALCITE-2018
> URL: https://issues.apache.org/jira/browse/CALCITE-2018
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> *Problem description*
> When rootLogger level is DEBUG, unit tests
> * MaterializationTest.testMaterializationSubstitution2
> * MaterializationTest.testJoinMaterializationUKFK8
> * MaterializationTest.testJoinMaterializationUKFK6
> * JdbcTest.testWhereNot
> unit tests are failed with error AssertionError: rel has lower cost than best
> cost of subset.
> Full stack trace for test
> {{MaterializationTest.testMaterializationSubstitution2}}:
> {noformat}
> java.lang.AssertionError: rel
> [rel#245:EnumerableUnion.ENUMERABLE.[](input#0=rel#246:Subset#5.ENUMERABLE.[],input#1=rel#239:Subset#6.ENUMERABLE.[0],all=true)]
> has lower cost {14.0 rows, 19.0 cpu, 0.0 io} than best cost {15.0 rows, 20.0
> cpu, 0.0 io} of subset [rel#243:Subset#7.ENUMERABLE.[]]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.validate(VolcanoPlanner.java:906)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:866)
> at
> org.apache.calcite.plan.volcano.Vo
[jira] [Comment Edited] (CALCITE-2018) Queries failed with AssertionError: rel has lower cost than best cost of subset
[
https://issues.apache.org/jira/browse/CALCITE-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300355#comment-16300355
]
Volodymyr Vysotskyi edited comment on CALCITE-2018 at 12/22/17 5:06 PM:
[~jcamachorodriguez], considering rels in RelSubset, I observed cases where
such metadata as row count is different for few rels from the same RelSet. So I
don't think that we can be sure that some kind of metadata left unchanged.
Currently, I am reworking this fix.
Main points that already done:
0) Left the same metadata cache structure as it was in my original PR. It
allows finding and deleting all cached values for the same RelNode easier.
1) Made changes to use the same {{RelOptCluster}} instance for RelNodes, so it
makes possible to use single {{RelMetadataQuery}} instance between the first
call of {{RelOptCluster.getMetadataQuery()}} and
{{RelOptCluster.invalidateMetadataQuery()}} call.
2) Cached metadata values are removed for current {{RelSubset}} and all parent
{{RelNode}} s when {{RelSubset.propagateCostImprovements()}} is called and best
rel node was changed.
3) Cached metadata values are removed for current {{RelNode}} when its input
was changed.
4) Made changes in {{RelSet.mergeWith()}} method to call
{{RelSubset.propagateCostImprovements()}} when resulting best is known instead
of just assigning {{RelSubset.best}}. It is needed for the cases when
{{getOrCreateSubset()}} method returns existing {{RelSubset}}, so we need to
inform its parent rels that its best rel node was changed. (Currently, only
otherSubset parents are informed.)
Currently, left single unresolved issue:
When {{RelNode}} instance is passed into
{{RelSubset.propagateCostImprovements()}} method, its parent subset is taken
from the {{VolcanoPlanner.mapRel2Subset}} map. If exists another subset, whose
{{RelTraitSet}} is satisfied by {{RelNode}} instance {{RelTraitSet}},
{{VolcanoPlanner.validate()}} method throws an error.
was (Author: vvysotskyi):
[~jcamachorodriguez], considering rels in RelSubset, I observed cases where
such metadata as row count is different for few rels from the same RelSet. So I
don't think that we can be sure that some kind of metadata left unchanged.
Currently, I am reworking this fix.
Main points that already done:
0) Left the same metadata cache structure as it was in my original PR. It
allows finding and deleting all cached values for the same RelNode easier.
1) Made changes to use the same {{RelOptCluster}} instance for RelNodes, so it
makes possible to use single {{RelMetadataQuery}} instance between the first
call of {{RelOptCluster.getMetadataQuery()}} and
{{RelOptCluster.invalidateMetadataQuery()}} call.
2) Cached metadata values are removed for current {{RelSubset}} and all parent
{{RelNode}} s when {{RelSubset.propagateCostImprovements()}} is called and best
rel node was changed.
3) Cached metadata values are removed for current {{RelNode}} when its input
was changed.
4) Made changes in {{RelSet.mergeWith()}} method to call
{{RelSubset.propagateCostImprovements()}} when resulting best is known instead
of just assigning {{RelSubset.best}}. It is needed for the cases when
{{getOrCreateSubset()}} method returns existing {{RelSubset}}, so we need to
inform its parent rels that its best rel node was changed. (Currently, only
otherSubset parents are informed.)
Currently, left single unresolved issue:
When {{RelNode}} instance is passed into
{{RelSubset.propagateCostImprovements()}} method, and its {{RelTraitSet}} does
not satisfies subset {{RelTraitSet}}, {{RelSubset}} does not tries to compare
its cost with best cost. Later, if the {{RelTraitSet}} is changed for the same
{{RelNode}} instance, and it satisfies subset {{RelTraitSet}},
{{VolcanoPlanner.validate()}} method throws an error.
> Queries failed with AssertionError: rel has lower cost than best cost of
> subset
> ---
>
> Key: CALCITE-2018
> URL: https://issues.apache.org/jira/browse/CALCITE-2018
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> *Problem description*
> When rootLogger level is DEBUG, unit tests
> * MaterializationTest.testMaterializationSubstitution2
> * MaterializationTest.testJoinMaterializationUKFK8
> * MaterializationTest.testJoinMaterializationUKFK6
> * JdbcTest.testWhereNot
> unit tests are failed with error AssertionError: rel has lower cost than best
> cost of subset.
> Full stack trace for test
> {{MaterializationTest.testMaterializationSubstitution2}}:
> {noformat}
> java.lang.AssertionError: rel
> [rel#245:EnumerableUnion.ENUMERABLE.[](input#0=rel#246:Subset#5.E
