Re: Join, SemiJoin, Correlate

2019-04-17 Thread Yuzhao Chen
hub.com > > > > [3] > > https://community.apache.org/apache-way/apache-project-maturity-model.html#quality > > > > > > > > > On Apr 14, 2019, at 14:53, Walaa Eldin Moustafa > > > > > > > wrote: > > > > > > > > >

Re: Join, SemiJoin, Correlate

2019-04-17 Thread Ruben Q L
ty-model.html#quality > >> > >>> On Apr 14, 2019, at 14:53, Walaa Eldin Moustafa > wrote: > >>> > >>> Agreed, but not sure what would the best way to do it be without > >>> making the code very confusing. > >>> > >>

Re: Join, SemiJoin, Correlate

2019-04-15 Thread Julian Hyde
3A%2F%2Fgithub.com >> [3] >> https://community.apache.org/apache-way/apache-project-maturity-model.html#quality >> >>> On Apr 14, 2019, at 14:53, Walaa Eldin Moustafa >>> wrote: >>> >>> Agreed, but not sure what would the best way to do it be wi

Re: Join, SemiJoin, Correlate

2019-04-15 Thread Yuzhao Chen
ot sure what would the best way to do it be without > > making the code very confusing. > > > > On Sat, Apr 13, 2019 at 2:46 PM Haisheng Yuan > > wrote: > > > > > > I share the same concern with you. > > > > > > > > > > > > >

Re: Join, SemiJoin, Correlate

2019-04-14 Thread Hongze Zhang
e the same concern with you. >> >> >> >> >> >> Thanks~ >> Haisheng >> Yuan---------- >> 发件人:Stamatis Zampetakis >> 日 期:2019年04月14日 05:37:29 >> 收件人: >> 主 题:Re: Join, SemiJoin, Correlate >&g

Re: Join, SemiJoin, Correlate

2019-04-14 Thread Walaa Eldin Moustafa
anks~ > Haisheng > Yuan-- > 发件人:Stamatis Zampetakis > 日 期:2019年04月14日 05:37:29 > 收件人: > 主 题:Re: Join, SemiJoin, Correlate > > Hi Danny, > > Thanks a lot for taking this on, it is a great start! > > I didn't look thoroughly through the PR but I noticed that there a

Re: Join, SemiJoin, Correlate

2019-04-13 Thread YuZhao Chen
Thx, Stamatis and Haisheng Yuan I did rename EnumerableCorrelate and EnumerableJoin, your suggestions are reasonable, I would add them back and mark as deprecated. For Correlate and LogicalCorrelate i already keep the old constructor and method #create(). I will recheck if there are some APIs

Re: Join, SemiJoin, Correlate

2019-04-13 Thread YuZhao Chen
Stamatis and Haisheng Yuan I did rename EnumerableCorrelate and EnumerableJoin, your suggestions are reasonable, I would add them back and mark as deprecated. For Correlate and LogicalCorrelate i already kept the old constructor and method #create(). I will recheck if there are some APIs

Re: Join, SemiJoin, Correlate

2019-04-13 Thread Stamatis Zampetakis
Hi Danny, Thanks a lot for taking this on, it is a great start! I didn't look thoroughly through the PR but I noticed that there are many renaming/refactoring of public APIs. I am not sure if we should introduce so many breaking changes without prior notice. A most conservative approach would be

Re: Join, SemiJoin, Correlate

2019-04-12 Thread Yuzhao Chen
Hi, @Haisheng Yuan, @Julian Hyde, @Stamatis Zampetakis, @Walaa Eldin Moustafa I have did the work for this discussion, and look forward to your suggestions. ### Diff - Deprecate SemiJoin, EquiJoin, EnumerableSemiJoin, SemiJoinType, EnumerableSemiJoinRule, EnumerableThetaJoin - Make

Re: Join, SemiJoin, Correlate

2019-04-01 Thread Walaa Eldin Moustafa
Correlate (or Apply) can be > > > transformed into a Join, or a Join > > > > is transfomed into a Correlate (Or Apply), in case there is an index > can > > > be used in inner relation. > > > > > > > > What I am not comfortable with is: > > > > In SQL Server and GPDB Orca optimzier, Sql

Re: Join, SemiJoin, Correlate

2019-04-01 Thread Yuzhao Chen
nslated into logical > > relation as it should be ( > > > keep subquery as it is), then use all kinds of apply rules to unnest > > subqueries based on cost model, > > > which seems reasonable to me. > > > But in Calcite, we can not only decorrelate in S

Re: Join, SemiJoin, Correlate

2019-04-01 Thread Stamatis Zampetakis
me. > > But in Calcite, we can not only decorrelate in SqlToRel stage, but also > can do it in SubqueryRemoveRule. > > Should we unify them all in the rules and keep SqlToRelConverter simple? > > > > Thanks ~ > > Haisheng Yuan > > --

Re: Join, SemiJoin, Correlate

2019-03-25 Thread Julian Hyde
t; Haisheng Yuan > ------ > 发件人:Stamatis Zampetakis > 日 期:2019年03月23日 07:31:35 > 收件人: > 主 题:Re: Join, SemiJoin, Correlate > > Since we are discussing this topic I thought it would be could to bring > back > to the

Re: Join, SemiJoin, Correlate

2019-03-25 Thread Julian Hyde
ubPlan (in Postgres’ term) is a Postgres physical relational node to >>> evaluate correlated subquery. What I mean is correlated subquery that can’t >>> be decorrelated can’t be implemented by hashjoin or mergejoin. But it is >>> off topic. >>> >>> Tha

Re: Re: Join, SemiJoin, Correlate

2019-03-25 Thread Haisheng Yuan
, but also can do it in SubqueryRemoveRule. Should we unify them all in the rules and keep SqlToRelConverter simple? Thanks ~ Haisheng Yuan -- 发件人:Stamatis Zampetakis 日 期:2019年03月23日 07:31:35 收件人: 主 题:Re: Join, SemiJoin, Correlate Since we

Re: Join, SemiJoin, Correlate

2019-03-25 Thread Michael Mior
> > > Thanks ~ > > Haisheng Yuan > > ---------- > > 发件人:Walaa Eldin Moustafa > > 日 期:2019年03月21日 09:31:41 > > 收件人: > > 抄 送:Stamatis Zampetakis > > 主 题:Re: Re: Join, SemiJoin, Correlate > > >

Re: Join, SemiJoin, Correlate

2019-03-23 Thread YuZhao Chen
y. What I mean is correlated subquery that can’t > be decorrelated can’t be implemented by hashjoin or mergejoin. But it is > off topic. > > > > Thanks ~ > > Haisheng Yuan > > ---------- > > 发件人:Walaa El

Re: Join, SemiJoin, Correlate

2019-03-22 Thread Stamatis Zampetakis
ive because it lets Join, SemiJoin > and Correlate continue to be structurally different. > > >> > > >> Julian > > >> > > >> > > >> > > >> > > >>> On Mar 20, 2019, at 6:55 PM, Haisheng Yuan > wrote: > > >>>

Re: Join, SemiJoin, Correlate

2019-03-21 Thread Walaa Eldin Moustafa
t;>> > >>> SubPlan (in Postgres’ term) is a Postgres physical relational node to > >>> evaluate correlated subquery. What I mean is correlated subquery that > >>> can’t be decorrelated can’t be implemented by hashjoin or mergejoin. But > >>>

Re: Join, SemiJoin, Correlate

2019-03-21 Thread Julian Hyde
subquery that can’t >>> be decorrelated can’t be implemented by hashjoin or mergejoin. But it is >>> off topic. >>> >>> Thanks ~ >>> Haisheng Yuan >>> -- >>>

Re: Join, SemiJoin, Correlate

2019-03-21 Thread Walaa Eldin Moustafa
can’t be implemented by hashjoin or mergejoin. But it is > > off topic. > > > > Thanks ~ > > Haisheng Yuan > > ---------- > > 发件人:Walaa Eldin Moustafa > > 日 期:2019年03月21日 09:31:41 > > 收件人: > > 抄 送:Stamatis Zampetakis > > 主 题:Re: Re: Join,

Re: Join, SemiJoin, Correlate

2019-03-21 Thread Julian Hyde
> 收件人: > 抄 送:Stamatis Zampetakis > 主 题:Re: Re: Join, SemiJoin, Correlate > > Agreed with Stamatis. Currently: 1) Correlate is tied to IN, EXISTS, > NOT IN, NOT EXISTS, and 2) is used as an equivalent to nested loops > join. The issues here are: 1) IN, EXISTS, NOT IN, NOT EX

Re: Re: Re: Join, SemiJoin, Correlate

2019-03-20 Thread Haisheng Yuan
-- 发件人:Walaa Eldin Moustafa 日 期:2019年03月21日 09:31:41 收件人: 抄 送:Stamatis Zampetakis 主 题:Re: Re: Join, SemiJoin, Correlate Agreed with Stamatis. Currently: 1) Correlate is tied to IN, EXISTS, NOT IN, NOT EXISTS, and 2) is used as an equivalent to nested loops join. The issues

Re: Re: Join, SemiJoin, Correlate

2019-03-20 Thread Walaa Eldin Moustafa
> Thanks ~ > Haisheng Yuan > -- > 发件人:Stamatis Zampetakis > 日 期:2019年03月21日 07:13:15 > 收件人: > 主 题:Re: Join, SemiJoin, Correlate > > I have bumped into this quite a few times and I think we should really try &

Re: Re: Join, SemiJoin, Correlate

2019-03-20 Thread Haisheng Yuan
in Calcite, or SubPlan in Postgres’s terminology, but it can’t be implemented as HashJoin or MergeJoin. Thanks ~ Haisheng Yuan -- 发件人:Stamatis Zampetakis 日 期:2019年03月21日 07:13:15 收件人: 主 题:Re: Join, SemiJoin, Correlate I have bumped

Re: Join, SemiJoin, Correlate

2019-03-20 Thread Stamatis Zampetakis
I have bumped into this quite a few times and I think we should really try to improve the design of the join hierarchy. >From a logical point of view I think it makes sense to have the following operators: InnerJoin, LeftOuterJoin, FullOuterJoin, SemiJoin, AntiJoin, (GroupJoin) Yet I have not