This makes me another question, since Storm SQL should traverse the
selected plan so having similar requirement.
How we can visit RelSubset if it's an input node in current RelNode?
Before asking a question I was following 'best' rel in RelSubset, assuming
that best is what Volcano planner picks.
If it's not, which is recommended way to traverse RelNode tree?
Or if it is, we can just pick 'best' in RelSubset.explain() (only if
available) to represent selected plan, like I said before.
- Jungtaek Lim (HeartSaVioR)
On Fri, 14 Oct 2016 at 8:34 AM Julian Hyde <jh...@apache.org> wrote:
> I agree, it would be useful to be able to print out the plan (or the best
> plan known at the moment). I suppose RelWriter could have a method that
> chooses which member of a RelSubset is to be preferred; and if that member
> is the best according to some cost model then what will come out is the
> best plan.
>
> RelNode.buildCheapestPlan basically does this, but it would be nice to
> have a way print the plan non-destructively.
>
> Julian
>
>
> > On Oct 13, 2016, at 4:26 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
> >
> > Sorry I was not clear on that. RelSubSet is correct, and while executing
> > RelOptUtil.toString(), it calls RelSubSet.explain(), and
> > RelSubSet.explain() picks input with that manner.
> >
> > Yes I'm saying about after planning. I agree that RelOptUtil.toString()
> can
> > be called before planning so it can't handle.
> > If toString() doesn't care about that I'd like to see the possibility to
> > have a feature which prints out actual plan, not just showing structure
> of
> > rel. As one of end-user I would like to see the actual plan which has
> more
> > meaning and what I was expecting.
> >
> > 2016년 10월 14일 (금) 오전 8:15, Julian Hyde <jh...@apache.org>님이 작성:
> >
> >> No, it’s not possible to print the “actual plan” because until planning
> is
> >> finished there is no plan. The job of toString is to print the
> STRUCTURE of
> >> the rel, and cost should not come into that.
> >>
> >> There is no “RelSub” class - I presume you mean RelSubSet? But RelSubSet
> >> has no toString method (other than the one it inherits from
> >> AbstractRelNode). So I assume you are talking about
> >> RelOptUtil.toString(RelNode)?
> >>
> >> Julian
> >>
> >>
> >>> On Oct 13, 2016, at 4:05 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
> >>>
> >>> Thanks Julian for the quick response. I'll follow CALCITE-794.
> >>>
> >>> And I got more details:
> >>>
> >>> rels:
> >>> 0 = {LogicalFilter@3579}
> >>>
> >>
> "rel#10:LogicalFilter.NONE.[](input=rel#9:Subset#0.ENUMERABLE.[],condition=>($0,
> >>> 3))"
> >>> 1 = {LogicalProject@3584}
> >>>
> >>
> "rel#12:LogicalProject.NONE.[](input=rel#11:Subset#1.NONE.[],ID=$0,NAME=$1,ADDR=$2)"
> >>> 2 = {StormProjectRel@3554}
> >>>
> >>
> "rel#21:StormProjectRel.STORM_LOGICAL.[](input=rel#20:Subset#1.STORM_LOGICAL.[],ID=$0,NAME=$1,ADDR=$2)"
> >>> 3 = {LogicalCalc@3585}
> >>>
> >>
> "rel#22:LogicalCalc.NONE.[[]](input=rel#11:Subset#1.NONE.[],expr#0..2={inputs},ID=$t0,NAME=$t1,ADDR=$t2)"
> >>> 4 = {TridentStormFilterRel@3569}
> >>>
> >>
> "rel#24:TridentStormFilterRel.STORM_LOGICAL.[](input=rel#23:Subset#0.STORM_LOGICAL.[],condition=>($0,
> >>> 3))"
> >>> 5 = {TridentStormCalcRel@3586}
> >>>
> >>
> "rel#25:TridentStormCalcRel.STORM_LOGICAL.[[]](input=rel#20:Subset#1.STORM_LOGICAL.[],expr#0..2={inputs},ID=$t0,NAME=$t1,ADDR=$t2)"
> >>> 6 = {LogicalCalc@3587}
> >>>
> >>
> "rel#26:LogicalCalc.NONE.[[]](input=rel#9:Subset#0.ENUMERABLE.[],expr#0..2={inputs},expr#3=3,expr#4=>($t0,
> >>> $t3),ID=$t0,NAME=$t1,ADDR=$t2,$condition=$t4)"
> >>> 7 = {TridentStormCalcRel@3588}
> >>>
> >>
> "rel#27:TridentStormCalcRel.STORM_LOGICAL.[[]](input=rel#23:Subset#0.STORM_LOGICAL.[],expr#0..2={inputs},expr#3=3,expr#4=>($t0,
> >>> $t3),ID=$t0,NAME=$t1,ADDR=$t2,$condition=$t4)"
> >>>
> >>> best:
> >>>
> >>
> rel#24:TridentStormFilterRel.STORM_LOGICAL.[](input=rel#23:Subset#0.STORM_LOGICAL.[],condition=>($0,
> >>> 3))
> >>>
> >>> Relsub.toString() just picks the first one, which might not be same as
> >>> Volcano planner selects. (Please correct me if I'm wrong.)
> >>> If it is, I think it's not intuitive though they're logically
> equivalent,
> >>> because when user requests explain user wants to see actual (selected)
> >>> plan. Is it possible to provide the actual plan? If 'best' representing
> >> the
> >>> selection, why not just printing out the best?
> >>>
> >>> Btw, I filed an issue:
> >> https://issues.apache.org/jira/browse/CALCITE-1438
> >>> I'm not clearly understanding on your solution for now, but if you
> >> haven't
> >>> had time to resolve after I followed up CALCITE-794, I'll try to make a
> >>> patch.
> >>>
> >>> Thanks,
> >>> Jungtaek Lim (HeartSaVioR)
> >>>
> >>> 2016년 10월 14일 (금) 오전 7:30, Julian Hyde <jh...@apache.org>님이 작성:
> >>>
> >>>> Cycles in the rel graph are difficult to avoid. See
> >>>> https://issues.apache.org/jira/browse/CALCITE-794 <
> >>>> https://issues.apache.org/jira/browse/CALCITE-794> for details. They
> >> are
> >>>> not fatal for optimization (as long as the nodes in the graph have
> >> positive
> >>>> cost, the cheapest plan (which is basically a path through the graph)
> >> will
> >>>> not be a cycle) but they are still best avoided. Adding
> simplifications
> >> in
> >>>> RelBuilder eliminates some common causes of cycles.
> >>>>
> >>>> I agree that RelOptUtil.toString must not give StackOverflowError. Can
> >> you
> >>>> please log a bug for this?
> >>>>
> >>>> The solution is probably straightforward: maintain a set of “active”
> >> nodes
> >>>> in the RelWriter created by RelOptUtil.toString.
> >>>>
> >>>> Julian
> >>>>
> >>>>
> >>>>> On Oct 13, 2016, at 3:17 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
> >>>>>
> >>>>> Hi devs,
> >>>>>
> >>>>> While I'm converting Storm SQL to convert Calcite logical to Storm's
> >> own
> >>>>> logical, I found one of Storm's unit test is failing. I put
> >>>>> RelOptUtil.toString() on every tests, and broken test is throwing
> >>>>> StackOverflowError.
> >>>>>
> >>>>> When it was also failing from IDEA, I dug it more, and found that one
> >> of
> >>>>> rels in Relsub has parent Relsub as 'input' (Relsub and Project).
> >>>>> Fortunately it was not selected to 'best', but Relsub print out first
> >>>>> occurence of rel which match the trait, and unfortunately it's first
> >> one.
> >>>>>
> >>>>> Query is really simple, INSERT INTO BAR SELECT ID, NAME, ADDR FROM
> FOO
> >>>>> WHERE ID > 3. It was not making an issue when Storm SQL uses Calcite
> >>>>> logical.
> >>>>>
> >>>>> I'm not sure making cross reference is a problem, but IMO throwing
> >>>>> StackOverflowError in explain is a major problem.
> >>>>>
> >>>>> I picked the workaround by printing out 'best' if it's available (not
> >>>> null)
> >>>>> instead of first occurrence of rel which match the trait.
> >>>>>
> >>>>> Does it make sense? If then I'll come up with filing an issue and
> >>>> following
> >>>>> pull request. I don't have an idea to reproduce so might not have
> test
> >> on
> >>>>> it.
> >>>>>
> >>>>> Thanks,
> >>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>
> >>>>> ps. Below is the ruleset I'm experimenting with. Please correct if
> I'm
> >>>>> using conflict rules together, or any odd things.
> >>>>>
> >>>>> SortRemoveRule.INSTANCE,
> >>>>> FilterToCalcRule.INSTANCE,
> >>>>> ProjectToCalcRule.INSTANCE,
> >>>>> FilterCalcMergeRule.INSTANCE,
> >>>>> ProjectCalcMergeRule.INSTANCE,
> >>>>> CalcMergeRule.INSTANCE,
> >>>>> PruneEmptyRules.FILTER_INSTANCE,
> >>>>> PruneEmptyRules.PROJECT_INSTANCE,
> >>>>> PruneEmptyRules.UNION_INSTANCE,
> >>>>> ProjectFilterTransposeRule.INSTANCE,
> >>>>> FilterProjectTransposeRule.INSTANCE,
> >>>>> ProjectRemoveRule.INSTANCE,
> >>>>> ReduceExpressionsRule.FILTER_INSTANCE,
> >>>>> ReduceExpressionsRule.PROJECT_INSTANCE,
> >>>>> ReduceExpressionsRule.CALC_INSTANCE,
> >>>>> UnionEliminatorRule.INSTANCE,
> >>>>> StormScanRule.INSTANCE,
> >>>>> StormFilterRule.INSTANCE,
> >>>>> StormProjectRule.INSTANCE,
> >>>>> StormAggregateRule.INSTANCE,
> >>>>> StormJoinRule.INSTANCE,
> >>>>> StormModifyRule.INSTANCE,
> >>>>> StormCalcRule.INSTANCE
> >>>>
> >>>>
> >>
> >>
>
>
