The VolcanoPlanner started to work after I changed the table to Scannable and
by explicitly changing to EnumerableConvention. Now, VolcanoPlanner object does
not do anything and I get the initial query plan back unmodified but
in EnumerableConvention.
This had worked just fine with HepPlanner.
*1. Initial LogicalPlan:*
LogicalProject(id=[$0], sum_units=[$1], sum_id=[+($1, $0)])
LogicalAggregate(group=[{0}], sum_units=[SUM($1)])
LogicalFilter(condition=[>($0, 1)])
LogicalUnion(all=[true])
LogicalProject(id=[$0], units=[$2])
LogicalTableScan(table=[[SALES, Orders1]])
LogicalProject(id=[$0], units=[$2])
*2. After changing the traitset:*
RelOptPlanner planner = loginalPlan
.getCluster().getPlanner().chooseDelegate();
RelTraitSet desiredTraits = loginalPlan
.getTraitSet().replace(EnumerableConvention.INSTANCE);
loginalPlan = planner.changeTraits(loginalPlan, desiredTraits);
AbstractConverter(subset=[rel#62:Subset#7.ENUMERABLE.[]],
convention=[ENUMERABLE], sort=[[]])
LogicalProject(subset=[rel#35:Subset#7.NONE.[]], id=[$0], sum_units=[$1],
sum_id=[+($1, $0)])
LogicalAggregate(subset=[rel#33:Subset#6.NONE.[]], group=[{0}],
sum_units=[SUM($1)])
LogicalFilter(subset=[rel#31:Subset#5.NONE.[]], condition=[>($0, 1)])
LogicalUnion(subset=[rel#29:Subset#4.NONE.[]], all=[true])
LogicalProject(subset=[rel#24:Subset#1.NONE.[]], id=[$0],
units=[$2])
LogicalTableScan(subset=[rel#22:Subset#0.NONE.[]],
table=[[SALES, Orders1]])
LogicalProject(subset=[rel#27:Subset#3.NONE.[]], id=[$0],
units=[$2])
LogicalTableScan(subset=[rel#25:Subset#2.NONE.[]],
table=[[SALES, Orders2]])
*3. Final running of Volcano planner:*
planner.addRule(FilterSetOpTransposeRule.INSTANCE);
planner.addRule(AggregateReduceFunctionsRule.INSTANCE);
planner.addRule(AggregateUnionTransposeRule.INSTANCE);
planner.setRoot(loginalPlan);
System.out.println(RelOptUtil.toString(planner.findBestExp()));
EnumerableProject(id=[$0], sum_units=[$1], sum_id=[+($1, $0)])
EnumerableAggregate(group=[{0}], sum_units=[$SUM0($1)])
EnumerableFilter(condition=[>($0, 1)])
EnumerableUnion(all=[true])
EnumerableProject(id=[$0], units=[$2])
EnumerableInterpreter
BindableTableScan(table=[[SALES, Orders1]])
EnumerableProject(id=[$0], units=[$2])
EnumerableInterpreter
BindableTableScan(table=[[SALES, Orders2]])
On Wed, Jun 22, 2016 at 8:13 PM, Zhangrucong <[email protected]> wrote:
> Hi Milinda,
>
> Thanks for your help.
>
> In Bindables.RULES, there is a BINDABLE_PROJECT_RULE for converting
> LogicalProject RelNode. So, I think it is not the reason of convention
> failure.
> When I make the test table class implement TranslatableTable and
> FilterableTable interface, volcano planner works well. In addition to
> FilterableTable, ProjectableFilterable and Scannable interfaces also fix
> that exception.
>
> But I have a little knowledge of the reason that the bindable rules
> require a translatable and filterable(or ProjectableFilterable, scannable)
> table.
> Could you give me an explanation?
>
> -----邮件原件-----
> 发件人: Milinda Pathirage [mailto:[email protected]]
> 发送时间: 2016年6月23日 0:14
> 收件人: [email protected]
> 主题: Re: [Calcite] Volcano planner fails to optimize
>
> Hi,
>
> I think this is caused by missing LogicalProject converter rule. You can
> figure which rules are missing by going through the planner state. The
> error message says 'Node [rel#11:Subset#2.BINDABLE.[]] could not be
> implemented'. That is because rel#11:Subset#2 has children whose inputs
> are still in NONE convention. For examples, rel#12:AbstractConverter's
> input 'rel#10:Subset#2' is still in NONE convention. This is because
> rel#9:LogicalDelta is missing its convention (or in NONE). Then if we look
> at the input to this delta, you can see that input's (rel#8:Subset#1)
> convention is also NONE. rel#8:Subset#1's convention is NONE because
> rel#7:LogicalProject has NONE convention.
>
> Thanks
> Milinda
>
> On Tue, Jun 21, 2016 at 11:33 PM, Zhangrucong <[email protected]>
> wrote:
>
> > Hi everyone,
> >
> > I want to test calcite optimization of volcano planner with null cost
> > factory. To avoid implementing some RelNodes and ConvertRules, I use
> > BindableConvention trait set to transform converted RelNode. The
> > QueryPlanner references to the implementation of SamzaSQL:
> >
> >
> > https://github.com/milinda/samza-sql/blob/master/samza-sql-planner/src
> > /main/java/org/apache/samza/sql/planner/QueryPlanner.java
> >
> > I replace the SamzaLogicalConvention.INSTANCE with the
> > BindableConvention.INSTANCE and remove the HepPlanner optimization.
> >
> > With the simple SQL test("select stream * from orders"), the volcano
> > planner always complains following errors:
> >
> > org.apache.calcite.plan.RelOptPlanner$CannotPlanException: Node
> > [rel#11:Subset#2.BINDABLE.[]] could not be implemented; planner state:
> >
> > Root: rel#11:Subset#2.BINDABLE.[]
> > Original rel:
> >
> > Sets:
> > Set#0, type: RecordType(INTEGER orderId, INTEGER productId)
> > rel#6:Subset#0.ENUMERABLE.[], best=rel#3,
> importance=0.7290000000000001
> > rel#3:EnumerableTableScan.ENUMERABLE.[](table=[orders]),
> > rowcount=100.0, cumulative cost={100.0 rows, 101.0 cpu, 0.0 io}
> > rel#20:Subset#0.BINDABLE.[], best=null,
> > importance=0.36450000000000005 Set#1, type: RecordType(INTEGER orderId,
> INTEGER productId)
> > rel#8:Subset#1.NONE.[], best=null, importance=0.81
> >
> >
> > rel#7:LogicalProject.NONE.[](input=rel#6:Subset#0.ENUMERABLE.[],orderI
> > d=$0,productId=$1),
> > rowcount=100.0, cumulative cost={inf}
> > rel#17:Subset#1.BINDABLE.[], best=null, importance=0.405
> >
> >
> > rel#16:InterpretableConverter.BINDABLE.[](input=rel#8:Subset#1.NONE.[]
> > ),
> > rowcount=100.0, cumulative cost={inf}
> >
> >
> > rel#21:BindableProject.BINDABLE.[](input=rel#20:Subset#0.BINDABLE.[],o
> > rderId=$0,productId=$1),
> > rowcount=100.0, cumulative cost={inf}
> > Set#2, type: RecordType(INTEGER orderId, INTEGER productId)
> > rel#10:Subset#2.NONE.[], best=null, importance=0.9
> > rel#9:LogicalDelta.NONE.[](input=rel#8:Subset#1.NONE.[]),
> > rowcount=100.0, cumulative cost={inf}
> > rel#11:Subset#2.BINDABLE.[], best=null, importance=1.0
> >
> >
> > rel#12:AbstractConverter.BINDABLE.[](input=rel#10:Subset#2.NONE.[],con
> > vention=BINDABLE,sort=[]),
> > rowcount=100.0, cumulative cost={inf}
> >
> >
> > rel#13:InterpretableConverter.BINDABLE.[](input=rel#10:Subset#2.NONE.[
> > ]),
> > rowcount=100.0, cumulative cost={inf}
> >
> > at
> >
> org.apache.calcite.plan.volcano.RelSubset$CheapestPlanReplacer.visit(RelSubset.java:443)
> > at
> >
> org.apache.calcite.plan.volcano.RelSubset.buildCheapestPlan(RelSubset.java:293)
> > at
> >
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:841)
> > at
> > org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:334)
> > at
> > org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:290)
> >
> > I have no idea what to do with the runtime exceptions. Anything did I
> > forget ? Or Did I misuse the BindableConvention trait set ?
> >
> > Thanks.
> >
> >
> >
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center Indiana
> University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
>
