Hi Milinda, I have another question about HepPlanner. When I add a list of rules to HepPlanner, I saw that the planner didn’t apply some rules inside above rule set from the log dumped by calcite. But after I change the order of some rules, moving them to the end of rule set, the HepPlanner don’t filter out them this time.
So, I don’t know which order to arrange the rules in rule set to ensure that they can be applied by HepPlanner successfully. Any tutorial for that? Thanks. Xinglu Yi 发件人: Milinda Pathirage [mailto:[email protected]] 发送时间: 2016年6月24日 23:55 收件人: [email protected] 抄送: yixinglu 主题: Re: 答复: [Calcite] Volcano planner fails to optimize Sorry for the mistake on my side. I totally forgot some rules may check for specific implementations of Table, etc. As I understand this has something to do with the Calcite interpreter implementation (Please have a look at org.apache.calcite.interpreter package). Interpreter calls 'scan' method of the table to read the rows. So this basically depends on how your query execution engine is implemented. Thanks Milinda On Wed, Jun 22, 2016 at 11:13 PM, Zhangrucong <[email protected]<mailto:[email protected]>> wrote: Hi Milinda, Thanks for your help. In Bindables.RULES, there is a BINDABLE_PROJECT_RULE for converting LogicalProject RelNode. So, I think it is not the reason of convention failure. When I make the test table class implement TranslatableTable and FilterableTable interface, volcano planner works well. In addition to FilterableTable, ProjectableFilterable and Scannable interfaces also fix that exception. But I have a little knowledge of the reason that the bindable rules require a translatable and filterable(or ProjectableFilterable, scannable) table. Could you give me an explanation? -----邮件原件----- 发件人: Milinda Pathirage [mailto:[email protected]<mailto:[email protected]>] 发送时间: 2016年6月23日 0:14 收件人: [email protected]<mailto:[email protected]> 主题: Re: [Calcite] Volcano planner fails to optimize Hi, I think this is caused by missing LogicalProject converter rule. You can figure which rules are missing by going through the planner state. The error message says 'Node [rel#11:Subset#2.BINDABLE.[]] could not be implemented'. That is because rel#11:Subset#2 has children whose inputs are still in NONE convention. For examples, rel#12:AbstractConverter's input 'rel#10:Subset#2' is still in NONE convention. This is because rel#9:LogicalDelta is missing its convention (or in NONE). Then if we look at the input to this delta, you can see that input's (rel#8:Subset#1) convention is also NONE. rel#8:Subset#1's convention is NONE because rel#7:LogicalProject has NONE convention. Thanks Milinda On Tue, Jun 21, 2016 at 11:33 PM, Zhangrucong <[email protected]<mailto:[email protected]>> wrote: > Hi everyone, > > I want to test calcite optimization of volcano planner with null cost > factory. To avoid implementing some RelNodes and ConvertRules, I use > BindableConvention trait set to transform converted RelNode. The > QueryPlanner references to the implementation of SamzaSQL: > > > https://github.com/milinda/samza-sql/blob/master/samza-sql-planner/src > /main/java/org/apache/samza/sql/planner/QueryPlanner.java > > I replace the SamzaLogicalConvention.INSTANCE with the > BindableConvention.INSTANCE and remove the HepPlanner optimization. > > With the simple SQL test("select stream * from orders"), the volcano > planner always complains following errors: > > org.apache.calcite.plan.RelOptPlanner$CannotPlanException: Node > [rel#11:Subset#2.BINDABLE.[]] could not be implemented; planner state: > > Root: rel#11:Subset#2.BINDABLE.[] > Original rel: > > Sets: > Set#0, type: RecordType(INTEGER orderId, INTEGER productId) > rel#6:Subset#0.ENUMERABLE.[], best=rel#3, importance=0.7290000000000001 > rel#3:EnumerableTableScan.ENUMERABLE.[](table=[orders]), > rowcount=100.0, cumulative cost={100.0 rows, 101.0 cpu, 0.0 io} > rel#20:Subset#0.BINDABLE.[], best=null, > importance=0.36450000000000005 Set#1, type: RecordType(INTEGER orderId, > INTEGER productId) > rel#8:Subset#1.NONE.[], best=null, importance=0.81 > > > rel#7:LogicalProject.NONE.[](input=rel#6:Subset#0.ENUMERABLE.[],orderI > d=$0,productId=$1), > rowcount=100.0, cumulative cost={inf} > rel#17:Subset#1.BINDABLE.[], best=null, importance=0.405 > > > rel#16:InterpretableConverter.BINDABLE.[](input=rel#8:Subset#1.NONE.[] > ), > rowcount=100.0, cumulative cost={inf} > > > rel#21:BindableProject.BINDABLE.[](input=rel#20:Subset#0.BINDABLE.[],o > rderId=$0,productId=$1), > rowcount=100.0, cumulative cost={inf} > Set#2, type: RecordType(INTEGER orderId, INTEGER productId) > rel#10:Subset#2.NONE.[], best=null, importance=0.9 > rel#9:LogicalDelta.NONE.[](input=rel#8:Subset#1.NONE.[]), > rowcount=100.0, cumulative cost={inf} > rel#11:Subset#2.BINDABLE.[], best=null, importance=1.0 > > > rel#12:AbstractConverter.BINDABLE.[](input=rel#10:Subset#2.NONE.[],con > vention=BINDABLE,sort=[]), > rowcount=100.0, cumulative cost={inf} > > > rel#13:InterpretableConverter.BINDABLE.[](input=rel#10:Subset#2.NONE.[ > ]), > rowcount=100.0, cumulative cost={inf} > > at > org.apache.calcite.plan.volcano.RelSubset$CheapestPlanReplacer.visit(RelSubset.java:443) > at > org.apache.calcite.plan.volcano.RelSubset.buildCheapestPlan(RelSubset.java:293) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:841) > at > org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:334) > at > org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:290) > > I have no idea what to do with the runtime exceptions. Anything did I > forget ? Or Did I misuse the BindableConvention trait set ? > > Thanks. > > > -- Milinda Pathirage PhD Student | Research Assistant School of Informatics and Computing | Data to Insight Center Indiana University twitter: milindalakmal skype: milinda.pathirage blog: http://milinda.pathirage.org -- Milinda Pathirage PhD Student | Research Assistant School of Informatics and Computing | Data to Insight Center Indiana University twitter: milindalakmal skype: milinda.pathirage blog: http://milinda.pathirage.org
