Hi Jesús, Thank you very much for the response, I will check this out.
Best, Victor On Tue, Feb 23, 2016 at 2:27 PM, Jesus Camacho Rodriguez < [email protected]> wrote: > Hi Victor, > > The time taken to generate the optimize plan will depend on the size of > the plan and the rewriting rules that you are applying. > > You will need to explore where the time is spent (too many firings of some > rules, on the logic of the planer itself, etc.) > > Thanks, > Jesús > > > > On 2/19/16, 1:51 PM, "Victor Giannakouris - Salalidis" < > [email protected]> wrote: > > >Milinda, Jesus > > > >Thank you very much for your help, I followed your advice and now > >everything seems to work well. But the problem I have now is that in case > >of adding many rules the planning is very slow. May you know why is that > >occurring? > > > >Best, > >Victor > > > >On Thu, Feb 18, 2016 at 8:21 PM, Jesus Camacho Rodriguez < > >[email protected]> wrote: > > > >> Hi Victor, and welcome, > >> > >> I checked the code you attached and it seems you did not add any rules > to > >> the HepPlanner. > >> You should manually add the rules that you want to trigger, as Milinda > >> pointed out, for instance: > >> hepPlanner.addRule(ReduceExpressionsRule.CALC_INSTANCE); > >> hepPlanner.addRule(ProjectToWindowRule.PROJECT); > >> > >> > >> Thanks, > >> Jesús > >> > >> > >> > >> > >> > >> > >> On 2/18/16, 6:52 PM, "Milinda Pathirage" <[email protected]> wrote: > >> > >> >Hi Victor, > >> > > >> >I don't know much about how planning works internally. But it is > possible > >> >that some rule were not applied. I think you can verify this by > creating a > >> >HepPlanner with relevant rules and sending your plan through that. Note > >> >that I'm not 100% sure whether this will work or not. > >> > > >> >HepProgramBuilder hepProgramBuilder = new HepProgramBuilder(); > >> >hepProgramBuilder.addRuleClass(ReduceExpressionsRule.class); > >> >hepProgramBuilder.addRuleClass(ProjectToWindowRule.class); > >> >HepPlanner hepPlanner = new HepPlanner(hepProgramBuilder.build()); > >> >hepPlanner.addRule(ReduceExpressionsRule.CALC_INSTANCE); > >> >hepPlanner.addRule(ProjectToWindowRule.PROJECT); > >> > > >> >hepPlanner.setRoot(convertedNode); > >> > > >> >RelNode rel = hepPlanner.findBestExp(); > >> > > >> >You should customize above to add rules you need for your scenario. > >> > > >> >Thanks > >> >Milinda > >> > > >> >On Thu, Feb 18, 2016 at 2:00 AM, Victor Giannakouris - Salalidis < > >> >[email protected]> wrote: > >> > > >> >> Hello. I am trying to implement a planner in order to generate > optimal > >> >> logical query plans using some statistics I provide to the schema. > >> >> Currently, the only available statistics is the number of rows of > each > >> >> table. > >> >> > >> >> I am using HepPlanner. My actual problem is that when the > >> *findBestExp()* > >> >> is called, the resulting plan is not optimized. That is, the query is > >> just > >> >> parsed and the join order is the same as the one I provide in the > input > >> >> query, neither filter push downs are being applied. > >> >> > >> >> For example, for the query > >> >> > >> >> "SELECT * FROM ftable f, products p WHERE f.id = p.pid AND p.pid = > 2" > >> >> > >> >> the resulting plan is: > >> >> > >> >> 12:LogicalProject(id=[$0], desc=[$1], price=[$2], loc=[$3], pid=[$4], > >> >> pdesc=[$5]): rowcount = 225000.0, cumulative cost = 1.05002E7 > >> >> 10:LogicalFilter(condition=[AND(=($0, $4), =($4, 2))]): rowcount = > >> >> 225000.0, cumulative cost = 1.02752E7 > >> >> 8:LogicalJoin(condition=[true], joinType=[inner]): rowcount = > 1.0E7, > >> >> cumulative cost = 1.00502E7 > >> >> 0:EnumerableTableScan(table=[[fTable]]): rowcount = 50000.0, > >> >> cumulative cost = 50000.0 > >> >> 1:EnumerableTableScan(table=[[products]]): rowcount = 200.0, > >> >> cumulative cost = 200.0 > >> >> > >> >> I implemented this using Hive's TestCBORuleFiredOnlyOnce.java > >> >> < > >> >> > >> > https://github.com/apache/hive/blob/48b201ee163252b2127ce04fbf660df70312888a/ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java > >> >> > > >> >> and PlannerImpl.java > >> >> < > >> >> > >> > https://github.com/apache/calcite/blob/5323d8d48baa2d7bc8dea8b03bc0bda93563e0f9/core/src/main/java/org/apache/calcite/prepare/PlannerImpl.java > >> >> > > >> >> as examples and there are some classes or overrided methods which I > >> >> currently use as “black boxes”. Here is a link with the code of my > basic > >> >> class: http://pastebin.com/HysfNa8S. > >> >> > >> >> -- > >> >> Victor Giannakouris - Salalidis > >> >> > >> >> LinkedIn: > >> >> http://gr.linkedin.com/pub/victor-giannakouris-salalidis/69/585/b23/ > >> >> Personal Page: http://gsvic.github.io > >> >> > >> > > >> > > >> > > >> >-- > >> >Milinda Pathirage > >> > > >> >PhD Student | Research Assistant > >> >School of Informatics and Computing | Data to Insight Center > >> >Indiana University > >> > > >> >twitter: milindalakmal > >> >skype: milinda.pathirage > >> >blog: http://milinda.pathirage.org > >> > -- Victor Giannakouris - Salalidis LinkedIn: http://gr.linkedin.com/pub/victor-giannakouris-salalidis/69/585/b23/ Personal Page: http://gsvic.github.io
