[jira] [Created] (CALCITE-3149) CACHE in RelDataTypeFactoryImpl can't be garbage collected

[Haisheng Yuan (JIRA)]

Haisheng Yuan created CALCITE-3149:
--

 Summary: CACHE in RelDataTypeFactoryImpl can't be garbage 
collected 
 Key: CALCITE-3149
 URL: https://issues.apache.org/jira/browse/CALCITE-3149
 Project: Calcite
  Issue Type: Bug
Reporter: Haisheng Yuan
 Attachments: image-2019-06-26-22-01-50-965.png

Key and Value are pointing to the same object, with soft references, the item 
in CACHE will not be garbage collected. And it is causing the RelDataType is 
not reused.

 !image-2019-06-26-22-01-50-965.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CALCITE-3148) Validator throws IndexOutOfBoundsException for SqlInsert when source and sink have non-equal fields number

[Danny Chan (JIRA)]

Danny Chan created CALCITE-3148:
---

 Summary: Validator throws IndexOutOfBoundsException for SqlInsert 
when source and sink have non-equal fields number
 Key: CALCITE-3148
 URL: https://issues.apache.org/jira/browse/CALCITE-3148
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.20.0
Reporter: Danny Chan
Assignee: Danny Chan
 Fix For: 1.21.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Modify Calcite Planner in Hive to remove GROUP BY

[Vineet Garg]

Hi Julian,

You are right it should produce zero rows not NULL. Thanks for the
correction.

Vineet


On Wed, Jun 26, 2019 at 4:49 PM Julian Hyde  wrote:

> > Select count(*) from empty_table group by  will produce NULL
>
> Really? I thought it should produce zero rows.
>
> Hsqldb:
>
> > select count(*) from "foodmart"."days" where false group by true;
> +-+
> |   C1|
> +-+
> +-+
> No rows selected (0.001 seconds)
>
>
> Julian
>
>
> > On Jun 26, 2019, at 1:12 PM, Vineet Garg  wrote:
> >
> > Hello Krzysztof,
> >
> > The rewrite you mention in Hive was done in HIVE-19674
> >  to be able to push
> such
> > group by to Druid. Currently there is no way to disable this rewrite.
> >
> > As for removing Group by , there are rules/rewrites which can
> > reduce grouping keys by removing constants but removing whole group by is
> > not safe since it can lead to semantically different query.
> > e.g. Select count(*) from empty_table group by  will produce
> NULL
> > but Select count(*) from empty_table will produce 0.
> >
> > P.S. There was a bug in HIVE-19674' patch which was further fixed by
> > HIVE-21539 .
> >
> > Regards,
> > Vineet Garg
> >
> > On Wed, Jun 26, 2019 at 7:08 AM Haisheng Yuan 
> > wrote:
> >
> >> Calcite has the rule that does the work. But you can't remove the group
> by
> >> clause if the constant is the only group key. The semantic is different
> >> without group key. Try it on empty relation, you will see the
> difference.
> >>
> >>
> >>
> >>
> >>
> >> Thanks~
> >> Haisheng
> >> Yuan--
> >> 发件人：Krzysztof Zarzycki
> >> 日 期：2019年06月26日 21:52:41
> >> 收件人：
> >> 主 题：Modify Calcite Planner in Hive to remove GROUP BY 
> >>
> >> Hello,
> >>
> >> While the question I have might look like regards to Hive, I believe is
> >> more about Calcite. I need to add a Calcite plan rule to Hive, that
> removes
> >> "Group by" clause when it groups by some constant value (GROUP BY TRUE
> more
> >> precisely). As far as I believe, the query semantically is the same.
> >> Could anyone on this mailing list help me how to do it properly? While
> I'm
> >> an experienced java engineer, I have no clue how to achieve this.
> >> I was trying to modify hive code to do this myself, but unfortunately I
> got
> >> only NullPointerExceptions.
> >>
> >>
> >> More context below:
> >> I want to use JdbcStorageHandler in Hive, that connects to Apache Kylin
> and
> >> forward queries there. Then I put Tableau on top of Hive. Unfortunately,
> >> the queries produced by Tableau to Hive and then reproduced by Calcite
> >> Planner to Kylin, cannot be handled by Kylin (which BTW uses Calcite as
> >> well). I disabled some of the hive optimizations which fixed some of my
> >> queries. But I'm stuck on one I cannot disable. Tableau generates a
> query
> >> with "GROUP BY 1.00...01" , that is translated to "GROUP BY TRUE",
> by
> >> Hive/Calcite. But neither of those can be handled by Kylin. I got an
> idea
> >> that I will remove GROUP BY completely, because in my understanding it's
> >> unecessary.
> >>
> >> I will be very grateful for your help,
> >> Kind Regards,
> >> Krzysztof
> >>
> >>
>
>

Re: Giving the Calcite logo some love

[Michael Mior]

The easiest approach is to convert the fonts to curves. Whatever
editing software you use should be able to do this, but I agree with
Julian that it's not critical. Thanks for working on this!
--
Michael Mior
mm...@apache.org

Le mer. 26 juin 2019 à 18:45, Stamatis Zampetakis  a écrit :
>
> I did my best to fix the problems with the fonts but I have no other OS at
> the moment to see if the problems disappeared.
> I also added two new variants (5G and 5H) based on the suggestions so far.
>
> Apologies for the multiple emails but I was not meant to be a designer :)
>
> On Wed, Jun 26, 2019 at 7:52 PM Ruben Q L  wrote:
>
> > My top two are: 5B, 2C
> >
> > Best regards,
> > Ruben
> >
> >
> > Le mer. 26 juin 2019 à 12:01, Stamatis Zampetakis  a
> > écrit :
> >
> > > FYI: I checked the logos today on Windows and Debian and the fonts for a
> > > couple of them (4A, 4B, 5B, 5C, 5D, 5E, 5F) are quite messed up.
> > Apparently
> > > they appear as they were supposed to only on MacOS.
> > >
> > > On Wed, Jun 26, 2019 at 4:10 AM Chunwei Lei 
> > > wrote:
> > >
> > > > I prefer the design of 5b too, but with the font of 2c.
> > > >
> > > > Best,
> > > > Chunwei
> > > >
> > > >
> > > > On Wed, Jun 26, 2019 at 9:09 AM Danny Chan 
> > wrote:
> > > >
> > > > > Top 2 in descending order:
> > > > > 1) 2C - for simple and elegant icon
> > > > > 2) 4D- for its nice color and font
> > > > >
> > > > > Best,
> > > > > Danny Chan
> > > > > 在 2019年6月26日 +0800 AM6:16，Stamatis Zampetakis  > >，写道：
> > > > > > Here is the updated link with the candidate logos:
> > > > > >
> > > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md
> > > > > >
> > > > > > Let's follow Julian's suggestion and start the nomination process
> > for
> > > > the
> > > > > > next 48 hours.
> > > > > >
> > > > > > Personally, the ones that I like the most are:
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5b
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5e
> > > > > >
> > > > > > They are exactly the same just with different colours which makes
> > > sense
> > > > > if
> > > > > > we want to use them in different places:
> > > > > > 5b would look nice in presentations while 5e would fit well with
> > the
> > > > > > existing colors of our website.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Jun 25, 2019 at 11:48 PM Julian Hyde 
> > > wrote:
> > > > > >
> > > > > > > OK, feel free to add the others to index.md (cherry-pick my
> > changes
> > > > > > > back to your branch).
> > > > > > >
> > > > > > > On Tue, Jun 25, 2019 at 2:36 PM Stamatis Zampetakis <
> > > > zabe...@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Actually there are more than six since "v" stands for variant
> > > > rather
> > > > > than
> > > > > > > > version.
> > > > > > > > I think it is worth including the rest since in various cases
> > > some
> > > > > of the
> > > > > > > > final variants did not end-up so well.
> > > > > > > >
> > > > > > > > On Tue, Jun 25, 2019 at 11:26 PM Julian Hyde  > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > I've put those 6 logos on one page.
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://github.com/julianhyde/calcite/blob/calcite-logo/site/img/index.md
> > > > > > > > >
> > > > > > > > > On Tue, Jun 25, 2019 at 1:19 PM Michael Mior <
> > mm...@apache.org
> > > >
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Below are the logos Stamatis kindly collected on his
> > branch.
> > > > > Since
> > > > > > > > > > there are only six, perhaps we can just proceed with a
> > vote?
> > > > > Although
> > > > > > > > > > I'm fine if we want to go through the nomination process as
> > > > > well. I
> > > > > > > > > > like your point Julian that we can still make final tweaks
> > > once
> > > > > we
> > > > > > > > > > decide on a logo, so whatever the process is, we should
> > keep
> > > > > that in
> > > > > > > > > > mind.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt1-v3.svg
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt2-v3.svg
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt3-v1.svg
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt4-v4.svg
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt5-v6.svg
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > 

Re: Modify Calcite Planner in Hive to remove GROUP BY

[Julian Hyde]

> Select count(*) from empty_table group by  will produce NULL

Really? I thought it should produce zero rows.

Hsqldb:

> select count(*) from "foodmart"."days" where false group by true;
+-+
|   C1|
+-+
+-+
No rows selected (0.001 seconds)


Julian


> On Jun 26, 2019, at 1:12 PM, Vineet Garg  wrote:
> 
> Hello Krzysztof,
> 
> The rewrite you mention in Hive was done in HIVE-19674
>  to be able to push such
> group by to Druid. Currently there is no way to disable this rewrite.
> 
> As for removing Group by , there are rules/rewrites which can
> reduce grouping keys by removing constants but removing whole group by is
> not safe since it can lead to semantically different query.
> e.g. Select count(*) from empty_table group by  will produce NULL
> but Select count(*) from empty_table will produce 0.
> 
> P.S. There was a bug in HIVE-19674' patch which was further fixed by
> HIVE-21539 .
> 
> Regards,
> Vineet Garg
> 
> On Wed, Jun 26, 2019 at 7:08 AM Haisheng Yuan 
> wrote:
> 
>> Calcite has the rule that does the work. But you can't remove the group by
>> clause if the constant is the only group key. The semantic is different
>> without group key. Try it on empty relation, you will see the difference.
>> 
>> 
>> 
>> 
>> 
>> Thanks~
>> Haisheng
>> Yuan--
>> 发件人：Krzysztof Zarzycki
>> 日 期：2019年06月26日 21:52:41
>> 收件人：
>> 主 题：Modify Calcite Planner in Hive to remove GROUP BY 
>> 
>> Hello,
>> 
>> While the question I have might look like regards to Hive, I believe is
>> more about Calcite. I need to add a Calcite plan rule to Hive, that removes
>> "Group by" clause when it groups by some constant value (GROUP BY TRUE more
>> precisely). As far as I believe, the query semantically is the same.
>> Could anyone on this mailing list help me how to do it properly? While I'm
>> an experienced java engineer, I have no clue how to achieve this.
>> I was trying to modify hive code to do this myself, but unfortunately I got
>> only NullPointerExceptions.
>> 
>> 
>> More context below:
>> I want to use JdbcStorageHandler in Hive, that connects to Apache Kylin and
>> forward queries there. Then I put Tableau on top of Hive. Unfortunately,
>> the queries produced by Tableau to Hive and then reproduced by Calcite
>> Planner to Kylin, cannot be handled by Kylin (which BTW uses Calcite as
>> well). I disabled some of the hive optimizations which fixed some of my
>> queries. But I'm stuck on one I cannot disable. Tableau generates a query
>> with "GROUP BY 1.00...01" , that is translated to "GROUP BY TRUE", by
>> Hive/Calcite. But neither of those can be handled by Kylin. I got an idea
>> that I will remove GROUP BY completely, because in my understanding it's
>> unecessary.
>> 
>> I will be very grateful for your help,
>> Kind Regards,
>> Krzysztof
>> 
>> 

Re: Giving the Calcite logo some love

[Julian Hyde]

I’m not too worried about fonts. I assume that we can generate PNGs from the 
SVGs and those PNGs will render the same on all platforms. It makes our 
decision-making a little more complex, because people on Linux/Windows may not 
be seeing the exact product, but it’s good enough.

> On Jun 26, 2019, at 3:45 PM, Stamatis Zampetakis  wrote:
> 
> I did my best to fix the problems with the fonts but I have no other OS at
> the moment to see if the problems disappeared.
> I also added two new variants (5G and 5H) based on the suggestions so far.
> 
> Apologies for the multiple emails but I was not meant to be a designer :)
> 
> On Wed, Jun 26, 2019 at 7:52 PM Ruben Q L  wrote:
> 
>> My top two are: 5B, 2C
>> 
>> Best regards,
>> Ruben
>> 
>> 
>> Le mer. 26 juin 2019 à 12:01, Stamatis Zampetakis  a
>> écrit :
>> 
>>> FYI: I checked the logos today on Windows and Debian and the fonts for a
>>> couple of them (4A, 4B, 5B, 5C, 5D, 5E, 5F) are quite messed up.
>> Apparently
>>> they appear as they were supposed to only on MacOS.
>>> 
>>> On Wed, Jun 26, 2019 at 4:10 AM Chunwei Lei 
>>> wrote:
>>> 
 I prefer the design of 5b too, but with the font of 2c.
 
 Best,
 Chunwei
 
 
 On Wed, Jun 26, 2019 at 9:09 AM Danny Chan 
>> wrote:
 
> Top 2 in descending order:
> 1) 2C - for simple and elegant icon
> 2) 4D- for its nice color and font
> 
> Best,
> Danny Chan
> 在 2019年6月26日 +0800 AM6:16，Stamatis Zampetakis >> ，写道：
>> Here is the updated link with the candidate logos:
>> 
>>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md
>> 
>> Let's follow Julian's suggestion and start the nomination process
>> for
 the
>> next 48 hours.
>> 
>> Personally, the ones that I like the most are:
>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5b
>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5e
>> 
>> They are exactly the same just with different colours which makes
>>> sense
> if
>> we want to use them in different places:
>> 5b would look nice in presentations while 5e would fit well with
>> the
>> existing colors of our website.
>> 
>> 
>> 
>> On Tue, Jun 25, 2019 at 11:48 PM Julian Hyde 
>>> wrote:
>> 
>>> OK, feel free to add the others to index.md (cherry-pick my
>> changes
>>> back to your branch).
>>> 
>>> On Tue, Jun 25, 2019 at 2:36 PM Stamatis Zampetakis <
 zabe...@gmail.com
>> 
>>> wrote:
 
 Actually there are more than six since "v" stands for variant
 rather
> than
 version.
 I think it is worth including the rest since in various cases
>>> some
> of the
 final variants did not end-up so well.
 
 On Tue, Jun 25, 2019 at 11:26 PM Julian Hyde >> 
> wrote:
 
> I've put those 6 logos on one page.
> 
> 
>>> 
> 
 
>>> 
>> https://github.com/julianhyde/calcite/blob/calcite-logo/site/img/index.md
> 
> On Tue, Jun 25, 2019 at 1:19 PM Michael Mior <
>> mm...@apache.org
 
> wrote:
>> 
>> Below are the logos Stamatis kindly collected on his
>> branch.
> Since
>> there are only six, perhaps we can just proceed with a
>> vote?
> Although
>> I'm fine if we want to go through the nomination process as
> well. I
>> like your point Julian that we can still make final tweaks
>>> once
> we
>> decide on a logo, so whatever the process is, we should
>> keep
> that in
>> mind.
>> 
>> 
> 
>>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt1-v3.svg
>> 
> 
>>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt2-v3.svg
>> 
> 
>>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt3-v1.svg
>> 
> 
>>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt4-v4.svg
>> 
> 
>>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt5-v6.svg
>> 
> 
>>> 
> 
 
>>> 
>> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt6-v7.svg
>> 
>> --
>> Michael Mior
>> mm...@apache.org
>> 
>> Le mar. 25 juin 2019 à 15:04, Julian Hyde <
>> jh...@apache.org>
>>> a
>>> écrit :
>>> 
>>> About process. I propose the following:
>>> 
>>> Over the next 2 days, people can nominate one logo by
 replying
> to
>>> this
> email with the URL of their choice.
>>> 
>>> Michael then starts a vote 

Re: Giving the Calcite logo some love

[Stamatis Zampetakis]

I did my best to fix the problems with the fonts but I have no other OS at
the moment to see if the problems disappeared.
I also added two new variants (5G and 5H) based on the suggestions so far.

Apologies for the multiple emails but I was not meant to be a designer :)

On Wed, Jun 26, 2019 at 7:52 PM Ruben Q L  wrote:

> My top two are: 5B, 2C
>
> Best regards,
> Ruben
>
>
> Le mer. 26 juin 2019 à 12:01, Stamatis Zampetakis  a
> écrit :
>
> > FYI: I checked the logos today on Windows and Debian and the fonts for a
> > couple of them (4A, 4B, 5B, 5C, 5D, 5E, 5F) are quite messed up.
> Apparently
> > they appear as they were supposed to only on MacOS.
> >
> > On Wed, Jun 26, 2019 at 4:10 AM Chunwei Lei 
> > wrote:
> >
> > > I prefer the design of 5b too, but with the font of 2c.
> > >
> > > Best,
> > > Chunwei
> > >
> > >
> > > On Wed, Jun 26, 2019 at 9:09 AM Danny Chan 
> wrote:
> > >
> > > > Top 2 in descending order:
> > > > 1) 2C - for simple and elegant icon
> > > > 2) 4D- for its nice color and font
> > > >
> > > > Best,
> > > > Danny Chan
> > > > 在 2019年6月26日 +0800 AM6:16，Stamatis Zampetakis  >，写道：
> > > > > Here is the updated link with the candidate logos:
> > > > >
> > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md
> > > > >
> > > > > Let's follow Julian's suggestion and start the nomination process
> for
> > > the
> > > > > next 48 hours.
> > > > >
> > > > > Personally, the ones that I like the most are:
> > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5b
> > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5e
> > > > >
> > > > > They are exactly the same just with different colours which makes
> > sense
> > > > if
> > > > > we want to use them in different places:
> > > > > 5b would look nice in presentations while 5e would fit well with
> the
> > > > > existing colors of our website.
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Jun 25, 2019 at 11:48 PM Julian Hyde 
> > wrote:
> > > > >
> > > > > > OK, feel free to add the others to index.md (cherry-pick my
> changes
> > > > > > back to your branch).
> > > > > >
> > > > > > On Tue, Jun 25, 2019 at 2:36 PM Stamatis Zampetakis <
> > > zabe...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Actually there are more than six since "v" stands for variant
> > > rather
> > > > than
> > > > > > > version.
> > > > > > > I think it is worth including the rest since in various cases
> > some
> > > > of the
> > > > > > > final variants did not end-up so well.
> > > > > > >
> > > > > > > On Tue, Jun 25, 2019 at 11:26 PM Julian Hyde  >
> > > > wrote:
> > > > > > >
> > > > > > > > I've put those 6 logos on one page.
> > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/julianhyde/calcite/blob/calcite-logo/site/img/index.md
> > > > > > > >
> > > > > > > > On Tue, Jun 25, 2019 at 1:19 PM Michael Mior <
> mm...@apache.org
> > >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > Below are the logos Stamatis kindly collected on his
> branch.
> > > > Since
> > > > > > > > > there are only six, perhaps we can just proceed with a
> vote?
> > > > Although
> > > > > > > > > I'm fine if we want to go through the nomination process as
> > > > well. I
> > > > > > > > > like your point Julian that we can still make final tweaks
> > once
> > > > we
> > > > > > > > > decide on a logo, so whatever the process is, we should
> keep
> > > > that in
> > > > > > > > > mind.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt1-v3.svg
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt2-v3.svg
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt3-v1.svg
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt4-v4.svg
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt5-v6.svg
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt6-v7.svg
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Michael Mior
> > > > > > > > > mm...@apache.org
> > > > > > > > >
> > > > > > > > > Le mar. 25 juin 2019 à 15:04, Julian Hyde <
> jh...@apache.org>
> > a
> > > > > > écrit :
> > > > > > > > > >
> > > > > > > > > > About process. I propose the following:
> > > > > > > > > >
> > > > > > > > > > Over the next 2 days, people can nominate one logo by
> > > replying
> > > > to
> > > > > > this
> > > > > > > > email with the URL of their choice.
> > > > > > > > > >
> > > > > > > > > > Michael then 

Re: How to do SemiJoin?

[Chia-Hung Lin]

It looks like the operation is done by

1. Build main query
* `main query= builder.scan(table name).build`
2. Then push the main table
* `builder.push(main query)`
3. Build subquery
* `sub query = builder.scan().filter().build`
4. Push the subquery
* `builder.push(sub query)`
5. Then do semi-join operation on the condition field
* `builder.semiJoin(builder.equals(builder.filed(...filed name),
builder.filed(...field name)))`

Thanks for the suggestion, it's very helpful!

On Tue, 25 Jun 2019 at 23:26, Haisheng Yuan  wrote:
>
> The input parameter of semiJoin method should be join conditions, not just 
> fields.
> Check the example usage in RelMetadataTest.java#checkPredicates L1435.
>
> - Haisheng
>
> --
> 发件人：Chia-Hung Lin
> 日　期：2019年06月26日 05:38:30
> 收件人：
> 主　题：How to do SemiJoin?
>
> A quick question. I am trying to do semiJoin(). The code looks like below.
>
> val subquery = builder.
>   scan("foodmart", "customer").
>   filter(builder.equals(builder.field("city"), 
> builder.literal("Albany"))).
>   project(builder.field("customer_id")).
>   build
>
> val mainquery = builder.
>   scan("foodmart", "sales_fact_1998").
>   project(builder.field("unit_sales")).
>   build
>
> val relnode = builder.push(mainquery).push(subquery).semiJoin(
>   builder.field("customer_id")
> ).build
>
> But when executing, it throws an error message saying
> `java.lang.RuntimeException: java.sql.SQLException: Error while
> preparing statement [null]`, and following another exception that
> mentions
>
> Cause: org.codehaus.commons.compiler.CompileException: Line 18, Column
> 66: No applicable constructor/method found for actual parameters
> "java.math.BigDecimal"; candidates are: "public static boolean
> org.apache.calcite.runtime.SqlFunctions.isTrue(java.lang.Boolean)"
>
> Although it looks like code generation issue, I suspect it's more
> likely my incorrect usage to construct semiJoin function. What is the
> correct way to use semiJoin? Is there any example or doc that may
> contain related info? I have this question because I can not find such
> info after searching on the internet. Most of semiJoin results found
> on the internet are internal implementation which looks like not
> related to the usage.
>
> Thanks

Re: Modify Calcite Planner in Hive to remove GROUP BY

[Vineet Garg]

Hello Krzysztof,

The rewrite you mention in Hive was done in HIVE-19674
 to be able to push such
group by to Druid. Currently there is no way to disable this rewrite.

As for removing Group by , there are rules/rewrites which can
reduce grouping keys by removing constants but removing whole group by is
not safe since it can lead to semantically different query.
e.g. Select count(*) from empty_table group by  will produce NULL
but Select count(*) from empty_table will produce 0.

P.S. There was a bug in HIVE-19674' patch which was further fixed by
HIVE-21539 .

Regards,
Vineet Garg

On Wed, Jun 26, 2019 at 7:08 AM Haisheng Yuan 
wrote:

> Calcite has the rule that does the work. But you can't remove the group by
> clause if the constant is the only group key. The semantic is different
> without group key. Try it on empty relation, you will see the difference.
>
>
>
>
>
> Thanks~
> Haisheng
> Yuan--
> 发件人：Krzysztof Zarzycki
> 日 期：2019年06月26日 21:52:41
> 收件人：
> 主 题：Modify Calcite Planner in Hive to remove GROUP BY 
>
> Hello,
>
> While the question I have might look like regards to Hive, I believe is
> more about Calcite. I need to add a Calcite plan rule to Hive, that removes
> "Group by" clause when it groups by some constant value (GROUP BY TRUE more
> precisely). As far as I believe, the query semantically is the same.
> Could anyone on this mailing list help me how to do it properly? While I'm
> an experienced java engineer, I have no clue how to achieve this.
> I was trying to modify hive code to do this myself, but unfortunately I got
> only NullPointerExceptions.
>
>
> More context below:
> I want to use JdbcStorageHandler in Hive, that connects to Apache Kylin and
> forward queries there. Then I put Tableau on top of Hive. Unfortunately,
> the queries produced by Tableau to Hive and then reproduced by Calcite
> Planner to Kylin, cannot be handled by Kylin (which BTW uses Calcite as
> well). I disabled some of the hive optimizations which fixed some of my
> queries. But I'm stuck on one I cannot disable. Tableau generates a query
> with "GROUP BY 1.00...01" , that is translated to "GROUP BY TRUE", by
> Hive/Calcite. But neither of those can be handled by Kylin. I got an idea
> that I will remove GROUP BY completely, because in my understanding it's
> unecessary.
>
> I will be very grateful for your help,
> Kind Regards,
> Krzysztof
>
>

Re: Giving the Calcite logo some love

[Ruben Q L]

My top two are: 5B, 2C

Best regards,
Ruben


Le mer. 26 juin 2019 à 12:01, Stamatis Zampetakis  a
écrit :

> FYI: I checked the logos today on Windows and Debian and the fonts for a
> couple of them (4A, 4B, 5B, 5C, 5D, 5E, 5F) are quite messed up. Apparently
> they appear as they were supposed to only on MacOS.
>
> On Wed, Jun 26, 2019 at 4:10 AM Chunwei Lei 
> wrote:
>
> > I prefer the design of 5b too, but with the font of 2c.
> >
> > Best,
> > Chunwei
> >
> >
> > On Wed, Jun 26, 2019 at 9:09 AM Danny Chan  wrote:
> >
> > > Top 2 in descending order:
> > > 1) 2C - for simple and elegant icon
> > > 2) 4D- for its nice color and font
> > >
> > > Best,
> > > Danny Chan
> > > 在 2019年6月26日 +0800 AM6:16，Stamatis Zampetakis ，写道：
> > > > Here is the updated link with the candidate logos:
> > > >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md
> > > >
> > > > Let's follow Julian's suggestion and start the nomination process for
> > the
> > > > next 48 hours.
> > > >
> > > > Personally, the ones that I like the most are:
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5b
> > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5e
> > > >
> > > > They are exactly the same just with different colours which makes
> sense
> > > if
> > > > we want to use them in different places:
> > > > 5b would look nice in presentations while 5e would fit well with the
> > > > existing colors of our website.
> > > >
> > > >
> > > >
> > > > On Tue, Jun 25, 2019 at 11:48 PM Julian Hyde 
> wrote:
> > > >
> > > > > OK, feel free to add the others to index.md (cherry-pick my changes
> > > > > back to your branch).
> > > > >
> > > > > On Tue, Jun 25, 2019 at 2:36 PM Stamatis Zampetakis <
> > zabe...@gmail.com
> > > >
> > > > > wrote:
> > > > > >
> > > > > > Actually there are more than six since "v" stands for variant
> > rather
> > > than
> > > > > > version.
> > > > > > I think it is worth including the rest since in various cases
> some
> > > of the
> > > > > > final variants did not end-up so well.
> > > > > >
> > > > > > On Tue, Jun 25, 2019 at 11:26 PM Julian Hyde 
> > > wrote:
> > > > > >
> > > > > > > I've put those 6 logos on one page.
> > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://github.com/julianhyde/calcite/blob/calcite-logo/site/img/index.md
> > > > > > >
> > > > > > > On Tue, Jun 25, 2019 at 1:19 PM Michael Mior  >
> > > wrote:
> > > > > > > >
> > > > > > > > Below are the logos Stamatis kindly collected on his branch.
> > > Since
> > > > > > > > there are only six, perhaps we can just proceed with a vote?
> > > Although
> > > > > > > > I'm fine if we want to go through the nomination process as
> > > well. I
> > > > > > > > like your point Julian that we can still make final tweaks
> once
> > > we
> > > > > > > > decide on a logo, so whatever the process is, we should keep
> > > that in
> > > > > > > > mind.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt1-v3.svg
> > > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt2-v3.svg
> > > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt3-v1.svg
> > > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt4-v4.svg
> > > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt5-v6.svg
> > > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt6-v7.svg
> > > > > > > >
> > > > > > > > --
> > > > > > > > Michael Mior
> > > > > > > > mm...@apache.org
> > > > > > > >
> > > > > > > > Le mar. 25 juin 2019 à 15:04, Julian Hyde 
> a
> > > > > écrit :
> > > > > > > > >
> > > > > > > > > About process. I propose the following:
> > > > > > > > >
> > > > > > > > > Over the next 2 days, people can nominate one logo by
> > replying
> > > to
> > > > > this
> > > > > > > email with the URL of their choice.
> > > > > > > > >
> > > > > > > > > Michael then starts a vote between the top 3 (or the top 2,
> > if
> > > > > there
> > > > > > > are a lot of ties for 3rd place).
> > > > > > > > >
> > > > > > > > > After we have decided on a logo, we can still ask the
> author
> > to
> > > > > make
> > > > > > > final tweaks to that logo.
> > > > > > > > >
> > > > > > > > > Julian
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > On Jun 24, 2019, at 5:45 PM, Michael Mior <
> > mm...@apache.org>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Thanks Stamatis! So far I think I like pretty much every
> > > > > alternative
> > > > > > > > > > I've seen more than the current logo. I'm not sure what
> > > others
> > > > > think
> > > > > > > > > > about process, 

Re: Release 1.20 and release notes

[Kevin Risden]

Top level has src/ which has a bunch of config files for
checkstyle/forbidden-apis/etc. Might be a decent spot.

Kevin Risden


On Wed, Jun 26, 2019 at 11:07 AM Michael Mior  wrote:

> A Gradle task is great if we migrate to Gradle, but I don't think it's
> a compelling reason in itself. I would personally prefer to just have
> a copy of the script somewhere in the repo because the only purpose of
> putting it in the HOWTO would be so someone could copy and paste and
> run it. That said, I'm not sure where a good spot for such scripts to
> live would be in the folder hierarchy.
> --
> Michael Mior
> mm...@apache.org
>
> Le mar. 25 juin 2019 à 16:23, Julian Hyde  a écrit :
> >
> > Or just paste the script into HOWTO.
> >
> > It works fine on macOS, Linux and Windows/Cygwin, which I think covers
> it.
> >
> > Julian
> >
> >
> > > On Jun 25, 2019, at 1:19 PM, Vladimir Sitnikov <
> sitnikov.vladi...@gmail.com> wrote:
> > >
> > > Michael>If we're going to use it, I think we
> > > Michael>might as well have a copy in the Calcite repo.
> > >
> > > I have a plan here:
> > > 1) We migrate to Gradle
> > > 2) The script can be written as a Gradle task so it is easily
> available,
> > > and it works in all OSes
> > >
> > > Vladimir
> >
>

Re: Release 1.20 and release notes

[Michael Mior]

A Gradle task is great if we migrate to Gradle, but I don't think it's
a compelling reason in itself. I would personally prefer to just have
a copy of the script somewhere in the repo because the only purpose of
putting it in the HOWTO would be so someone could copy and paste and
run it. That said, I'm not sure where a good spot for such scripts to
live would be in the folder hierarchy.
--
Michael Mior
mm...@apache.org

Le mar. 25 juin 2019 à 16:23, Julian Hyde  a écrit :
>
> Or just paste the script into HOWTO.
>
> It works fine on macOS, Linux and Windows/Cygwin, which I think covers it.
>
> Julian
>
>
> > On Jun 25, 2019, at 1:19 PM, Vladimir Sitnikov 
> >  wrote:
> >
> > Michael>If we're going to use it, I think we
> > Michael>might as well have a copy in the Calcite repo.
> >
> > I have a plan here:
> > 1) We migrate to Gradle
> > 2) The script can be written as a Gradle task so it is easily available,
> > and it works in all OSes
> >
> > Vladimir
>

回复：Modify Calcite Planner in Hive to remove GROUP BY

[Haisheng Yuan]

Calcite has the rule that does the work. But you can't remove the group by 
clause if the constant is the only group key. The semantic is different without 
group key. Try it on empty relation, you will see the difference.





Thanks~
Haisheng Yuan--
发件人：Krzysztof Zarzycki
日　期：2019年06月26日 21:52:41
收件人：
主　题：Modify Calcite Planner in Hive to remove GROUP BY 

Hello,

While the question I have might look like regards to Hive, I believe is
more about Calcite. I need to add a Calcite plan rule to Hive, that removes
"Group by" clause when it groups by some constant value (GROUP BY TRUE more
precisely). As far as I believe, the query semantically is the same.
Could anyone on this mailing list help me how to do it properly? While I'm
an experienced java engineer, I have no clue how to achieve this.
I was trying to modify hive code to do this myself, but unfortunately I got
only NullPointerExceptions.


More context below:
I want to use JdbcStorageHandler in Hive, that connects to Apache Kylin and
forward queries there. Then I put Tableau on top of Hive. Unfortunately,
the queries produced by Tableau to Hive and then reproduced by Calcite
Planner to Kylin, cannot be handled by Kylin (which BTW uses Calcite as
well). I disabled some of the hive optimizations which fixed some of my
queries. But I'm stuck on one I cannot disable. Tableau generates a query
with "GROUP BY 1.00...01" , that is translated to "GROUP BY TRUE", by
Hive/Calcite. But neither of those can be handled by Kylin. I got an idea
that I will remove GROUP BY completely, because in my understanding it's
unecessary.

I will be very grateful for your help,
Kind Regards,
Krzysztof

Modify Calcite Planner in Hive to remove GROUP BY

[Krzysztof Zarzycki]

Hello,

While the question I have might look like regards to Hive, I believe is
more about Calcite. I need to add a Calcite plan rule to Hive, that removes
"Group by" clause when it groups by some constant value (GROUP BY TRUE more
precisely). As far as I believe, the query semantically is the same.
Could anyone on this mailing list help me how to do it properly? While I'm
an experienced java engineer, I have no clue how to achieve this.
I was trying to modify hive code to do this myself, but unfortunately I got
only NullPointerExceptions.


More context below:
I want to use JdbcStorageHandler in Hive, that connects to Apache Kylin and
forward queries there. Then I put Tableau on top of Hive. Unfortunately,
the queries produced by Tableau to Hive and then reproduced by Calcite
Planner to Kylin, cannot be handled by Kylin (which BTW uses Calcite as
well). I disabled some of the hive optimizations which fixed some of my
queries. But I'm stuck on one I cannot disable. Tableau generates a query
with "GROUP BY 1.00...01" , that is translated to "GROUP BY TRUE", by
Hive/Calcite. But neither of those can be handled by Kylin. I got an idea
that I will remove GROUP BY completely, because in my understanding it's
unecessary.

I will be very grateful for your help,
Kind Regards,
Krzysztof

[DISCUSSION] Unify throws in sql parser

[Danny Chan]

Now our parser has 3 kinds of throws behavior

[1] Use JavaCC generateParseException
[2] Use SqlUtil.newContextException
[3] Use JavaCC ParseExecption directly

For [1] and [2] there is a position info in the exception message, a throw may 
like:
> From line 1, column 15 to line 1, column 26:

But for 3, we only have the error message without pos info, which is not that 
user friendly when
the sql text is huge (there are 10 occurance in our parser).

So shall we unify them ? E.G. Use only 1 and 2 is enough for all the cases, the 
2 can totally replace 3.

Do you think this is necessary ?


[1] 
https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494
[2] 
https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386
[3] 
https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601

Best,
Danny Chan

Re: Giving the Calcite logo some love

[Stamatis Zampetakis]

FYI: I checked the logos today on Windows and Debian and the fonts for a
couple of them (4A, 4B, 5B, 5C, 5D, 5E, 5F) are quite messed up. Apparently
they appear as they were supposed to only on MacOS.

On Wed, Jun 26, 2019 at 4:10 AM Chunwei Lei  wrote:

> I prefer the design of 5b too, but with the font of 2c.
>
> Best,
> Chunwei
>
>
> On Wed, Jun 26, 2019 at 9:09 AM Danny Chan  wrote:
>
> > Top 2 in descending order:
> > 1) 2C - for simple and elegant icon
> > 2) 4D- for its nice color and font
> >
> > Best,
> > Danny Chan
> > 在 2019年6月26日 +0800 AM6:16，Stamatis Zampetakis ，写道：
> > > Here is the updated link with the candidate logos:
> > > https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md
> > >
> > > Let's follow Julian's suggestion and start the nomination process for
> the
> > > next 48 hours.
> > >
> > > Personally, the ones that I like the most are:
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5b
> > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/index.md#candidate-5e
> > >
> > > They are exactly the same just with different colours which makes sense
> > if
> > > we want to use them in different places:
> > > 5b would look nice in presentations while 5e would fit well with the
> > > existing colors of our website.
> > >
> > >
> > >
> > > On Tue, Jun 25, 2019 at 11:48 PM Julian Hyde  wrote:
> > >
> > > > OK, feel free to add the others to index.md (cherry-pick my changes
> > > > back to your branch).
> > > >
> > > > On Tue, Jun 25, 2019 at 2:36 PM Stamatis Zampetakis <
> zabe...@gmail.com
> > >
> > > > wrote:
> > > > >
> > > > > Actually there are more than six since "v" stands for variant
> rather
> > than
> > > > > version.
> > > > > I think it is worth including the rest since in various cases some
> > of the
> > > > > final variants did not end-up so well.
> > > > >
> > > > > On Tue, Jun 25, 2019 at 11:26 PM Julian Hyde 
> > wrote:
> > > > >
> > > > > > I've put those 6 logos on one page.
> > > > > >
> > > > > >
> > > >
> >
> https://github.com/julianhyde/calcite/blob/calcite-logo/site/img/index.md
> > > > > >
> > > > > > On Tue, Jun 25, 2019 at 1:19 PM Michael Mior 
> > wrote:
> > > > > > >
> > > > > > > Below are the logos Stamatis kindly collected on his branch.
> > Since
> > > > > > > there are only six, perhaps we can just proceed with a vote?
> > Although
> > > > > > > I'm fine if we want to go through the nomination process as
> > well. I
> > > > > > > like your point Julian that we can still make final tweaks once
> > we
> > > > > > > decide on a logo, so whatever the process is, we should keep
> > that in
> > > > > > > mind.
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt1-v3.svg
> > > > > > >
> > > > > >
> > > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt2-v3.svg
> > > > > > >
> > > > > >
> > > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt3-v1.svg
> > > > > > >
> > > > > >
> > > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt4-v4.svg
> > > > > > >
> > > > > >
> > > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt5-v6.svg
> > > > > > >
> > > > > >
> > > >
> >
> https://github.com/zabetak/calcite/blob/calcite-logo/site/img/logo-alt6-v7.svg
> > > > > > >
> > > > > > > --
> > > > > > > Michael Mior
> > > > > > > mm...@apache.org
> > > > > > >
> > > > > > > Le mar. 25 juin 2019 à 15:04, Julian Hyde  a
> > > > écrit :
> > > > > > > >
> > > > > > > > About process. I propose the following:
> > > > > > > >
> > > > > > > > Over the next 2 days, people can nominate one logo by
> replying
> > to
> > > > this
> > > > > > email with the URL of their choice.
> > > > > > > >
> > > > > > > > Michael then starts a vote between the top 3 (or the top 2,
> if
> > > > there
> > > > > > are a lot of ties for 3rd place).
> > > > > > > >
> > > > > > > > After we have decided on a logo, we can still ask the author
> to
> > > > make
> > > > > > final tweaks to that logo.
> > > > > > > >
> > > > > > > > Julian
> > > > > > > >
> > > > > > > >
> > > > > > > > > On Jun 24, 2019, at 5:45 PM, Michael Mior <
> mm...@apache.org>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > Thanks Stamatis! So far I think I like pretty much every
> > > > alternative
> > > > > > > > > I've seen more than the current logo. I'm not sure what
> > others
> > > > think
> > > > > > > > > about process, but I'd like to close discussion and take a
> > vote
> > > > in
> > > > > > the
> > > > > > > > > near future.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Michael Mior
> > > > > > > > > mm...@apache.org
> > > > > > > > >
> > > > > > > > > Le lun. 24 juin 2019 à 17:37, Stamatis Zampetakis <
> > > > zabe...@gmail.com>
> > > > > > a écrit :
> > > > > > > > > >
> > > > > > > > > > I added a couple of more alternatives under [1]. Let me
> >