[
https://issues.apache.org/jira/browse/CALCITE-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909448#comment-16909448
]
Volodymyr Vysotskyi commented on CALCITE-3257:
----------------------------------------------
I don't know the exact reason for adding calls to invalidate rel nodes after
the rule is applied, so I will operate only my assumptions.
I think that it was done to force recalculating rel node cost after the rule is
applied, so ideally we would have the correct cost for both unchanged and
updated rel nodes - when the rule is applied, child rel nodes were changed and
cost of current rel node was calculated considering children costs instead of
using the outdated cached value.
So ideally for the case when the tree wasn't changed, there is no need to
invalidate the cache, since cached values should be correct.
One of the issues, fixed in the scope of CALCITE-2018 was the case when several
instances caches are used in the same time - one cache instance was invalidated
after the rule is applied, but another one was used during the next rule
applying.
*As I see from the code, {{RuleQueue.dump()}} does not calculate rel nodes
cost...*
I think something similar happens in this case so invalidating the cache after
(or perhaps also before) calling {{dump()}} method will help to avoid this
issue, but the problem with coexisting several cache instances will still be
present.
Could you please check that CALCITE-2018 fixes the issue you have observed?
> Need to clear RelMetaDataQuery cache after dumping RelNodes and Graphviz
> ------------------------------------------------------------------------
>
> Key: CALCITE-3257
> URL: https://issues.apache.org/jira/browse/CALCITE-3257
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Xiening Dai
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> To repro -
> 1. Set Log4J log level to TRACE. So planner will dump rel node info every
> time at rule match
> (https://github.com/apache/calcite/blob/3124a85b93ff2f1b79484c7bd4cc41835d4f1920/core/src/main/java/org/apache/calcite/plan/volcano/RuleQueue.java#L435)
> 2. Run JdbcTest.testNotExistsCorrelated. Get below exception -
> java.lang.AssertionError: rel
> [rel#63:EnumerableAggregate.ENUMERABLE.[](input=RelSubset#62,group={0})] has
> lower cost {131.0 rows, 216.0 cpu, 0.0 io} than best cost {131.5 rows, 216.0
> cpu, 0.0 io} of subset [rel#60:Subset#4.ENUMERABLE.[]]
> at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.isValid(VolcanoPlanner.java:889)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:852)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:869)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1928)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:129)
> at
> org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:236)
> at
> org.apache.calcite.rel.convert.ConverterRule.onMatch(ConverterRule.java:141)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:286)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:314)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:522)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1466)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1398)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1464)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.explainMatches(CalciteAssert.java:1561)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.explainContains(CalciteAssert.java:1556)
> at
> org.apache.calcite.test.JdbcTest.testNotExistsCorrelated(JdbcTest.java:4562)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With
> materializationsEnabled=false, limit=0, sql=explain plan for select * from
> "hr"."emps" where not exists (
> select 1 from "hr"."depts"
> where "emps"."deptno"="depts"."deptno")
> at org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:268)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:554)
> ... 28 more
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
