's optimizer, which are Cascades framework based,
> > implemented the property enforcement in the core optimizer engine, not
> > through AbstractConverter and rules, physical operators just need to
> > implement those methods (or similar) I mentioned in email [1]. My goal is
> >
goal is
> completely abolishing AbstractConverter.
> >
> > [1]
> http://mail-archives.apache.org/mod_mbox/calcite-dev/201910.mbox/%3cd75b20f4-542a-4a73-897e-66ab426494c1.h.y...@alibaba-inc.com%3e
> >
> > - Haisheng
> >
> > --
> http://mail-archives.apache.org/mod_mbox/calcite-dev/201910.mbox/%3cd75b20f4-542a-4a73-897e-66ab426494c1.h.y...@alibaba-inc.com%3e
>
> - Haisheng
>
> --------------
> 发件人：Jinfeng Ni
> 日 期：2019年11月01日 14:10:30
> 收件人：
> 主 题：
.h.y...@alibaba-inc.com%3e
- Haisheng
--
发件人：Jinfeng Ni
日 期：2019年11月01日 14:10:30
收件人：
主 题：Re: [DISCUSS] On-demand traitset request
Hi Xiening,
"Let say if R and S doesn’t have sorting properties at all. In your
case, we would e
>>>>> SELECT DISTINCT c, b FROM
>>>>> ( SELECT R.c c, S.b b FROM R, S
>>>>> WHERE R.a=S.a and R.b=S.b and R.c=S.c) t;
>>>>>
>>>>> Suppose R is ordered by (c, b, a), and S is ordered by (b, c, a).
>>>>
physical merge join for the inner join, it may just
> >>> have collation sorted on a,b,c. Then the aggreate on top of join will
> >>> request another sort on c,b, thus we miss the best plan. What we
> >>> can do is requesting all the order combinations, which is n!,
/RelMdCollation.java
--Original Mail --
Sender:Haisheng Yuan
Send Date:Fri Oct 25 12:01:40 2019
Recipients:dev@calcite.apache.org (dev@calcite.apache.org)
Subject:Re: Re: Re: [DISCUSS] On-demand traitset request
I didn't say adding to RelNode, but a new API/interface
：Re: Re: [DISCUSS] On-demand traitset request
I have the same feeling, it seems to much interfaces for the physical node(we
do not really have physical class for physical nodes yet), so these interfaces
may just be put into the RelNode, that was too complex and to much for me, can
we have a way
; |-- TableScan on R
> > > +-- TableScan on S
> > > We may require the match type on S to be satisfy. (a,b) satisfies
> required
> > > distribution (a,b,c).
> > > Fot the outer child R, we require it to be exact match with inner.
> > >
> > > 5. ExecOrder getExecOrder()
> > > Returns how the operator's childre
gt; Returns how the operator's children is executed, left to right, or right
> > to left. Typically, hash join is right to left. We might use this as the
> > optimization order. To make sure we have correct plans, we have to optimize
> > child and enforce properties in the order that is specific to the physi
)
public T derivedTrait(RelTraitDef traitDef)
- Haisheng
--
发件人：Stamatis Zampetakis
日 期：2019年10月23日 14:53:38
收件人：
主 题：Re: Re: [DISCUSS] On-demand traitset request
Overall, I agree that better encapsulation of propagation and derivation
correct plans, we have to optimize
> child and enforce properties in the order that is specific to the physical
> operator.
> All the other dirty work should be done by the optimization engine, but
> not through rules, I believe. However, I havn't got any clear plan on how
> to
. However, I havn't got any clear plan on how to
achieve it inside the engine.
Haisheng
--
发件人：Jacques Nadeau
日 期：2019年10月21日 11:04:19
收件人：
主 题：Re: [DISCUSS] On-demand traitset request
Definitely agree that this has been a long
, S.b b FROM R, S
> > >>>WHERE R.a=S.a and R.b=S.b and R.c=S.c) t;
> > >>>
> > >>> Suppose R is ordered by (c, b, a), and S is ordered by (b, c, a).
> > >>> Here is the logical plan:
> > >>>
t;>> have collation sorted on a,b,c. Then the aggreate on top of join will
> >>> request another sort on c,b, thus we miss the best plan. What we
> >>> can do is requesting all the order combinations, which is n!, like
> >>> how the Values operator does. Bu
he above query, the Aggregate operator can derive
>>> possible traitsets that its child operator join can deliver, in which case,
>>> the possiple traitsets of join is
>>> 1. collation on (a,b,c) based on join condition,
>>> 2. collation on (c,b,a) based o
es can be reduced.
> > The DerivedTraitSets can be used to derive the possible traitsets from
> > Join, and pass through Project, Filter etc...
> >
> > This is just an example of non-distributed system, for distributed system,
> > it can save much more by considering
,
> because it is single-node system, but Hive/Flink are distributed system.
> - Haisheng
>
> --
> 发件人：Stamatis Zampetakis
> 日 期：2019年10月18日 14:53:41
> 收件人：
> 主 题：Re: [DISCUSS] On-demand traitset request
>
>
--
发件人：Stamatis Zampetakis
日 期：2019年10月18日 14:53:41
收件人：
主 题：Re: [DISCUSS] On-demand traitset request
Hi Haisheng,
This is an interesting topic but somehow in my mind I thought that this
mechanism is already in place.
When an operator (logical or physical) is created its
To clarify. The purpose of this API would be to give the search engine
more high-level as to the goals it should focus on. The performance
issues described in https://issues.apache.org/jira/browse/CALCITE-2970
seem to be due to the planner "trying everything", and the solution
might be to add a
Excellent, very important discussion. This has been a major missing
feature for a long time. Let's be sure to get to a conclusion and
implement something.
>From the Volcano paper:
"the search engine permits the optimizer implementor to specify
a number of physical property vectors to be
Hi Haisheng,
This is an interesting topic but somehow in my mind I thought that this
mechanism is already in place.
When an operator (logical or physical) is created its traitset is
determined in bottom-up fashion using the create
static factory method present in almost all operators. In my mind
TL;DR
Both top-down physical TraitSet request and bottom-up TraitSet
derivation have their strongth and weakness, we propose
on-demand TraitSet request to combine the above two, to reduce
the number of plan alternatives that are genereated, especially
in distributed system.
e.g.
select * from
23 matches
Mail list logo
