I think I found the origin of the exception. To convert a single query to a rel node I am using the PlannerImp provided by Frameworks.getPlanner(). This planner provides a VolcanoPlanner to the relnode clusters. When calling the HepPlanner.findBestExp() it will eventually call the applyTransformationResults which will compute a cost using the rel.getCluster().getPlanner() (which is a VolcanoPlanner).
Mark On Fri, 3 May 2019 at 19:56, Mark Pasterkamp <[email protected]> wrote: > I did not know this is how it works, I just copied the example above. > Would there be an easy way to create a RelNode containg a tablescan over > the materialized view "mv"? > Trying to create one using for instance a relbuilder gives a calcite > exception. > Otherwise I might just look for another test file in which I can get > access to the schema and use the materiaqlizationservice. > > Mark > > On Fri, 3 May 2019 at 18:35, Jesus Camacho Rodriguez > <[email protected]> wrote: > >> bq . Since we are talking about materialized views, I think in most cases >> tableRel should be simply a LogicalTableScan. >> >> Stamatis is correct about this, I had not realized tableRel == queryRel >> in >> your sample code. >> >> Thanks, >> Jesús >> >> On Fri, May 3, 2019 at 6:12 AM Stamatis Zampetakis <[email protected]> >> wrote: >> >> > I think the main problem comes from the fact that tableRel == queryRel >> in >> > the test case you provided. >> > Defining the materialized view like that basically says that when you >> find >> > a part of the query that satisfies queryRel replace it with itself. >> > In conjunction with the rule that is used, which allows partial >> rewritings >> > using union, you end up with a rule that matches infinite number of >> times. >> > Since we are talking about materialized views, I think in most cases >> > tableRel should be simply a LogicalTableScan. >> > The idea is that expression represented by queryRel is materialized >> into a >> > table so in order to retrieve the results we only need to scan the >> table. >> > >> > Regarding the "if (true)" statements that you observed, most likely they >> > were introduced as release toggles [1]. >> > However, since the last commit was in 2013 I think by now it is safe to >> > refactor that part and remove dead code. >> > >> > [1] https://www.martinfowler.com/articles/feature-toggles.html >> > >> > Best, >> > Stamatis >> > >> > On Fri, May 3, 2019 at 12:50 PM Mark Pasterkamp < >> > [email protected]> wrote: >> > >> > > Dear Jesus, >> > > >> > > I think your intuition in this regard is correct. >> > > After executing the main program in the HepPlanner the resulting plan >> > > contains a lot of circular references. >> > > Changing the matching order does not influence this behaviour. >> > > >> > > >> > > Mark >> > > >> > > On Thu, 2 May 2019 at 22:14, Jesus Camacho Rodriguez >> > > <[email protected]> wrote: >> > > >> > > > Mark, >> > > > >> > > > I have an intuition that this happens because the rule creates a >> > > partially >> > > > contained rewriting with a union, where one side contains a scan >> over >> > the >> > > > materialized view and the other side contains the query itself with >> a >> > > > filter on top excluding the data that is coming from the >> materialized >> > > view. >> > > > Then the rule is triggered on the plan representing the original >> query >> > > > again and the process is repeated. Have you tried changing the >> matching >> > > > order for your hep program? >> > > > >> > > > Thanks, >> > > > Jesús >> > > > >> > > > On Thu, May 2, 2019 at 8:53 AM Mark Pasterkamp < >> > > > [email protected]> >> > > > wrote: >> > > > >> > > > > Hi Stamatis, >> > > > > >> > > > > I have tried to recreate the issue but I have not been able to do >> > > that. I >> > > > > was however able to create a new exception which I don't quite >> > > > understand. >> > > > > The error happened when calcite was creating a union rewriting >> using >> > > > > materialized views. But trying to recreate this situation gave me >> > > another >> > > > > interesting one. >> > > > > This time, the planner rewrites one of the children nodes into >> > itself I >> > > > > would assume which causes a stack overflow. The method itself can >> be >> > > > found >> > > > > here: >> > > > > >> > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/mpasterkamp/calcite/blob/768b7928dbde5f6f9775a1119e7466d8eafafb4b/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java#L312 >> > > > > >> > > > > Perhaps I am doing something wrong, perhaps not? I am not >> > knowledgeable >> > > > > enough about this to understand why this is happening. Wish I >> could >> > > help >> > > > > more for that. >> > > > > >> > > > > Also, while investigating this issue I found another interesting >> > > artifact >> > > > > in de source code of the VolcanoCost. A lot of methods in this >> class >> > > have >> > > > > an "if (true)"-statement like here: >> > > > > >> > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/apache/calcite/blob/4b4d8037c5073e4eb5702b12bc4ecade31476616/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoCost.java#L100 >> > > > > >> > > > > Now I was just curious, is there any reason for this to be there >> that >> > > you >> > > > > know of? >> > > > > >> > > > > Thank you for responding and congratulations for your recent >> > > promotions. >> > > > > >> > > > > >> > > > > With kind regards, >> > > > > >> > > > > Mark >> > > > > >> > > > > >> > > > > On Thu, 2 May 2019 at 14:58, Stamatis Zampetakis < >> [email protected]> >> > > > > wrote: >> > > > > >> > > > > > Said like that it looks like a bug. >> > > > > > >> > > > > > I think the best would be to reproduce the exception as a unit >> test >> > > in >> > > > > > HepPlannerTest [1], RelOptRulesTest [2], or PlannerTest [3] so >> that >> > > we >> > > > > > could understand better the use case. >> > > > > > >> > > > > > [1] >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java >> > > > > > [2] >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java >> > > > > > [3] >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/tools/PlannerTest.java >> > > > > > >> > > > > > On Thu, May 2, 2019 at 7:56 AM Mark Pasterkamp < >> > > > > > [email protected]> >> > > > > > wrote: >> > > > > > >> > > > > > > I don't, I would assume that the HepPlanner.findBestExp() >> > > calculates >> > > > > the >> > > > > > > cost somewhere down the line >> > > > > > > >> > > > > > > On Thu, May 2, 2019, 03:31 Yuzhao Chen <[email protected]> >> > > wrote: >> > > > > > > >> > > > > > > > Why you care about cost when use HepPlanner ? The >> HepPlanner is >> > > > aimed >> > > > > > for >> > > > > > > > some deterministic planning rules, we usually do not need >> cost >> > in >> > > > > Hep. >> > > > > > > Some >> > > > > > > > exceptions like Join reorder may need a cost. >> > > > > > > > >> > > > > > > > What kind of planning promotion you did ? I'm kind of >> curious >> > > about >> > > > > it. >> > > > > > > > >> > > > > > > > Best, >> > > > > > > > Danny Chan >> > > > > > > > 在 2019年5月1日 +0800 PM9:27，Mark Pasterkamp < >> > > > > [email protected] >> > > > > > > > >，写道： >> > > > > > > > > Dear all, >> > > > > > > > > >> > > > > > > > > While playing around with the HepPlanner I ran into an >> issue >> > > > where >> > > > > > the >> > > > > > > > > planner wants to rewrite a query with a union rewrite. >> When >> > the >> > > > > > > > > RelMetaDataQuery computes the cost, the cost instance is a >> > > > > > VolcanoCost. >> > > > > > > > > Then when it tries to calculate the cost of one of the >> > union's >> > > > > > operands >> > > > > > > > it >> > > > > > > > > is a RelCostImpl which results in the ClassCastException. >> > > > > > > > > >> > > > > > > > > How would I go about solving this issue? As far as my >> > knowledge >> > > > > > goes, I >> > > > > > > > am >> > > > > > > > > not able to change the costhandler of the >> RelMetaDataQuery. >> > > > Another >> > > > > > > > > approach I could see is removing the cast in the >> VolcanoCost >> > > > class, >> > > > > > > but I >> > > > > > > > > would hope I do not have to do that. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > With kind regards, >> > > > > > > > > >> > > > > > > > > Mark >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
