Re: [DISCUSS] Drop unused branches from git repos

[Chunwei Lei]

+1 for the cleanup.

Best,
Chunwei


On Thu, Jan 26, 2023 at 9:05 AM Benchao Li  wrote:

> +1, thanks for the effort.
>
> Sergey Nuyanzin  于2023年1月23日周一 20:02写道：
>
> > +1 for  cleanup
> >
> > On Mon, Jan 23, 2023 at 9:53 AM Ruben Q L  wrote:
> >
> > > +1 for the cleanup.
> > >
> > > Best,
> > > Ruben
> > >
> > >
> > > On Mon, Jan 23, 2023 at 7:27 AM Alessandro Solimando <
> > > alessandro.solima...@gmail.com> wrote:
> > >
> > > > +1 from me as well, keeping unused branches around is only confusing.
> > > >
> > > > Best regards,
> > > > Alessandro
> > > >
> > > > On Sun 22 Jan 2023, 22:58 Julian Hyde, 
> wrote:
> > > >
> > > > > +1
> > > > >
> > > > > There’s probably a tag for each release but let’s make sure before
> we
> > > > drop
> > > > > the branch. If any branch just has a ‘prepare for next iteration’
> > > commit
> > > > > following the release tag I’m fine dropping that commit.
> > > > >
> > > > > Julian
> > > > >
> > > > >
> > > > > > On Jan 22, 2023, at 1:05 PM, Francis Chuang <
> > > francischu...@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > +1 for dropping them. We should do this for both calcite and
> > > > > calcite-avatica (there shouldn't be any dangling branches for
> > > > > calcite-avatica-go).
> > > > > >
> > > > > > On 23/01/2023 7:13 am, Stamatis Zampetakis wrote:
> > > > > >> Hi all,
> > > > > >> Apart from the main, site, and possibly one or two other
> branches,
> > > all
> > > > > the
> > > > > >> rest are not used by anyone unless I am missing something.
> > > > > >> Many of them (branch-X.Y) were generated during past releases
> but
> > > this
> > > > > >> pattern stopped some time ago.
> > > > > >> It is not a big deal keeping them around but it can be confusing
> > for
> > > > new
> > > > > >> comers (.e.g., we still have the master branch around there) and
> > > tools
> > > > > >> operating at repo level.
> > > > > >> The most recent example is the INFRA team informing us about the
> > use
> > > > of
> > > > > >> Travis in our repos I assume due to the presence of .travis.yml
> > file
> > > > in
> > > > > >> branch-X.Y branches.
> > > > > >> Are there any reasons for keeping them around? If not how about
> > > > dropping
> > > > > >> them.
> > > > > >> Best,
> > > > > >> Stamatis
> > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Sergey
> >
>
>
> --
>
> Best,
> Benchao Li
>

Re: [ANNOUNCE] New committer: Istvan Toth

[Chunwei Lei]

Congrats and welcome Istvan!

Best,
Chunwei


On Fri, Jan 27, 2023 at 12:58 AM Istvan Toth  wrote:

> Thanks for the warm welcome to everyone!
>
> A few words about myself:
>
> I am from Hungary, and I work for Cloudera.
> My focus is Apache Phoenix, and its sub-projects, but I also contribute to
> HBase and other projects at times.
>
> As many of you know, one of the Phoenix sub-projects is Phoenix Query
> Server (PQS), which is
> an essential part of the Phoenix ecosystem for many users.
>
> While working on PQS issues I quickly realized that PQS is just a very thin
> glue layer, and all the heavy lifting is in fact performed by Avatica,
> which led to me becoming active on the project and contributing to it.
>
> I am grateful to everyone in the community who helped with advice and
> reviews, especially to one of the project's founders Josh Elser.
> I was lucky to have him as a colleague, and have access to his thorough
> knowledge of Avatica.
>
> best regards
> Istvan
>
>
> On Thu, Jan 26, 2023 at 3:54 PM Michael Mior  wrote:
>
> > Welcome Istvan!
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> >
> > On Tue, Jan 24, 2023 at 4:58 PM Stamatis Zampetakis 
> > wrote:
> >
> > > Apache Calcite's Project Management Committee (PMC) has invited Istvan
> > > Toth to become a committer, and we are pleased to announce that he
> > > has accepted.
> > >
> > > Istvan began contributing to Avatica in 2019 and has since made notable
> > > contributions,
> > > particularly in the areas of authentication and connection handling.
> His
> > > efforts have
> > > helped improve the project's stability and security.
> > >
> > > Istvan, welcome, thank you for your contributions, and we look forward
> > > to your further interactions with the community! If you wish, feel free
> > to
> > > tell us more about yourself and what you are working on.
> > >
> > > As your first commit, please add yourself to the contributors list [1]
> > > and the community page will re-generate [2].
> > >
> > > Stamatis (on behalf of the Apache Calcite PMC)
> > >
> > > [1]
> > >
> https://github.com/apache/calcite/blob/main/site/_data/contributors.yml
> > >
> > > [2] https://calcite.apache.org/community/#project-members
> > >
> >
>
>
> --
> *István Tóth* | Sr. Staff Software Engineer
> *Email*: st...@cloudera.com
> cloudera.com 
> [image: Cloudera] 
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image: Cloudera
> on LinkedIn] 
> --
> --
>

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Chunwei Lei]

Congrats, Benchao!

Best,
Chunwei


On Fri, Jan 27, 2023 at 9:01 PM Michael Mior  wrote:

> Congratulations and welcome Benchao!
>
> --
> Michael Mior
> mm...@apache.org
>
>
> On Fri, Jan 27, 2023 at 4:51 AM Stamatis Zampetakis 
> wrote:
>
> > I am pleased to announce that Benchao has accepted an invitation to join
> > the Calcite PMC. Benchao has been a consistent and helpful figure in
> > the Calcite community for which we are very grateful. We look forward to
> > the continued contributions and support.
> >
> > Please join me in congratulating Benchao!
> >
> > Stamatis (on behalf of the Calcite PMC)
> >
>

Re: several JavaCC warnings "Choice conflict involving two expansions"

[Julian Hyde]

Good catch. Yes, we want to stay on top of these kinds of warnings.
(See broken windows theory [1].) Probably introduced by
https://issues.apache.org/jira/browse/CALCITE-5450. Probably missing
one or two LOOKAHEAD directives in the parser. Can you log a JIRA case
please.

Julian

[1] https://en.wikipedia.org/wiki/Broken_windows_theory


On Fri, Jan 27, 2023 at 9:50 AM Alessandro Solimando
 wrote:
>
> Hello everyone,
> while checking CI logs I have noticed that we have lots of JavaCC warnings
> related to ambiguous prefixes in the productions of one of our grammars.
>
> They also seem related to time functions, for which I have seen several
> related developments for BigQuery lately.
>
> Have we verified that our grammar is still behaving properly under this
> situation? Have we considered increasing the lookahead value as suggested?
> Shall we open a Jira ticket to have a closer look?
>
> Here is an example of CI logs showing the problem (although it is
> reproducible locally):
> https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull
>
>
> In what follows the extract that is relevant to the discussion at hand:
>
> > > Task :core:javaCCMain
> > Java Compiler Compiler Version 4.0 (Parser Generator)
> > (type "javacc" with no arguments for help)
> > Reading from file
> > /home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
> > . . .
> > Warning: Output directory
> > "/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
> > does not exist. Creating the directory.
> > Note: UNICODE_INPUT option is specified. Please make sure you create the
> > parser/lexer using a Reader with the correct character encoding.
> > Warning: Choice conflict involving two expansions at
> >  line 4930, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "MICROSECOND"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4931, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "MILLISECOND"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4936, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "DOW"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4937, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "DOY"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4938, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "ISODOW"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4939, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "ISOYEAR"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4940, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "WEEK"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4950, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "QUARTER"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4952, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "EPOCH"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4953, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "DECADE"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4954, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "CENTURY"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 4955, column 5 and line 4956, column 5 respectively.
> >  A common prefix is: "MILLENNIUM"
> >  Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> >  line 6549, column 9 and line 6551, column 9 respectively.
> >  A common prefix is: "WEEK" "("
> >  Consider using a lookahead of 3 or more for earlier expansion.
> > File "TokenMgrError.java" does not exist.  Will create one.
> > File "ParseException.java" 

Re: [SUPPORT] Semantics change when applying ConverterRule in CheapestPlanReplacer

[Julian Hyde]

Done. Let's continue the conversation at CALCITE-5503.

On Fri, Jan 27, 2023 at 10:08 AM Moritz Mack  wrote:
>
> Julian, I’ve created CALCITE-5503 as suggested.
>
> Would you be able to add me as a contributor in Jira and assign the ticket to 
> me.
> I opened a PR with the test case you’ve asked for and a rather trivial fix.
> https://github.com/apache/calcite/pull/3050
>
> Thanks,
> Moritz
>
> On 27.01.23, 10:00, "Moritz Mack"  wrote:
>
> Thanks a lot, Julian.
> I’ll look into that the next few days.
>
> /Moritz
>
> On 25.01.23, 17:04, "Julian Hyde"  wrote:
>
> It would seem a reasonable and useful enhancement if CheapestPlanReplacer did 
> some memoization. If it sees the same RelNode twice it should emit the same 
> result. I would be conservative, and memoize based on identity of the 
> RelNodes, not just equality.
>
> It would be useful if you could come up with a test case in pure Calcite, 
> then log a jira case.
>
> Julian
>
> > On Jan 25, 2023, at 12:56 AM, Moritz Mack  wrote:
> >
> > Dear Calcite Devs,
> >
> > I’m currently looking into an issue in the SQL extension [1] of Apache Beam 
> > and was hoping to find some advice here.
> > Using a bunch of Calcite ConverterRules [2], we convert a RelNode tree into 
> > a tree of BeamRelNodes which is then used to build a Beam DAG. Pretty 
> > standard I suppose …
> >
> > What I’m scratching my head about is that applying the converter rules 
> > changes the semantics of the graph, which it shouldn’t I thought. Or is 
> > that a wrong expectation?
> > Here’s a very simple SQL example to illustrate this (see also 
> > BeamUnionRelTest [3]):
> >
> > SELECT  order_id, site_id, price FROM ORDER_DETAILS
> > UNION ALL
> > SELECT  order_id, site_id, price FROM ORDER_DETAILS
> >
> > This is where we start at, the corresponding RelNode:
> >
> > LogicalUnion.NONE(input#0=LogicalProject#8,input#1=LogicalProject#10,all=true)
> >
> > Once the corresponding conversion rule [4] is applied to above by the 
> > CheapestPlanReplacer, but before visiting its old inputs, we get the 
> > following:
> >
> > BeamUnionRel.BEAM_LOGICAL(input#0=RelSubset#25,input#1=RelSubset#25,all=true)
> >
> > At this point both inputs refer to the same node (#25). However, once 
> > visiting the inputs [5] in CheapestPlanReplacer, that semantic information 
> > is lost as RelNodes get copied if inputs change [5].
> > In below result, the two inputs refer to different nodes:
> >
> > BeamUnionRel.BEAM_LOGICAL(input#0=BeamCalcRel#49,input#1=BeamCalcRel#50,all=true)
> >
> > This, however, currently prevents caching of intermediate results in the 
> > Beam DAG when this might be beneficial.
> >
> > Would you have any advice how to better approach this?
> > Of course, I could use a stateful copy operation to handle such repeated 
> > copy operations with the same parameters. But this seems wrong to be honest.
> >
> > Thanks a million!
> > Kind regards,
> > Moritz
> >
> >
> > [1] 
> > https://urldefense.com/v3/__https://github.com/apache/beam/issues/24314__;!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrH1HemJw$
> > [2] 
> > https://urldefense.com/v3/__https://github.com/apache/beam/blob/6ba647333c4c69fb6dfc65929456c7c11570382f/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/planner/BeamRuleSets.java*L136-L152__;Iw!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9Ubryge2rYo$
> > [3] 
> > https://urldefense.com/v3/__https://github.com/apache/beam/blob/6ba647333c4c69fb6dfc65929456c7c11570382f/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnionRelTest.java*L52-L71__;Iw!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrYLpRpco$
> > [4] 
> > 

Re: array literal in spark dialect

[Julian Hyde]

Thanks Guillaume. We'd love a fix. I saw you logged
https://issues.apache.org/jira/browse/CALCITE-5504 so let's continue
the conversation there.

On Fri, Jan 27, 2023 at 3:05 PM Guillaume Masse
 wrote:
>
> Array literals are unparsed incorrectly for the spark dialect.
>
> They should be in the form: array(1, 2, 3)
> However, they are unparsed as ARRAY[1, 2, 3]
>
> https://spark.apache.org/docs/latest/api/sql/#array
>
> I would be happy to submit a fix,
> Guillaume

array literal in spark dialect

[Guillaume Masse]

Array literals are unparsed incorrectly for the spark dialect.

They should be in the form: array(1, 2, 3)
However, they are unparsed as ARRAY[1, 2, 3]

https://spark.apache.org/docs/latest/api/sql/#array

I would be happy to submit a fix,
Guillaume

Re: [DISCUSS] Towards Calcite 1.33.0

[Julian Hyde]

The particular failure I was referring to was this one:

  > Task :druid:javadoc
  Picked up _JAVA_OPTIONS: -XX:GCTimeLimit=90 -XX:GCHeapFreeLimit=35
  error: Error fetching URL:
https://docs.oracle.com/en/java/javase/17/docs/api/
(java.net.SocketException: Connection reset)

in the  "macOS (JDK 18)" check [1] for the latest commit, c4155b8d. I
have never seen it before, but it looks like infrastructure.

CALCITE-5501 is a different issue, also intermittent, also not a
blocker for the release.

Julian

[1] https://github.com/apache/calcite/actions/runs/4020515767/jobs/6908529265

On Fri, Jan 27, 2023 at 2:22 PM Alessandro Solimando
 wrote:
>
> Hello,
> I think that Julian was referring to CALCITE-5501
>  for the intermittent
> CI issue, a PR fixing it is now available.
>
> I have no problem waiting for the release to be rolled out before merging
> it, just wanted to reach out to see if you prefer to merge it right away,
> in case please let me know.
>
> Btw, thanks Jess for being the release manager!
>
> Best regards,
> Alessandro
>
> On Fri, 27 Jan 2023 at 23:09, Jess Balint  wrote:
>
> > Thanks, will get the release notes PR up shortly.
> >
> > On Thu, Jan 26, 2023 at 7:29 PM Julian Hyde  wrote:
> >
> > > I have merged a fix for CALCITE-5489. Thanks to Tanner and TJ for their
> > > help.
> > >
> > > There is one failure in the build, but I'm pretty sure it is
> > > intermittent and is related to CI infrastructure, not our code.
> > >
> > > Jess, Go ahead and make that RC.
> > >
> > > Julian
> > >
> > >
> > > On Thu, Jan 26, 2023 at 10:50 AM Julian Hyde 
> > > wrote:
> > > >
> > > > I tried your PR https://github.com/apache/calcite/pull/3048/files. It
> > > fixes the bug, but the test cases you have included to not reproduce the
> > > bug. I think that TIMESTAMP_DIFF(quarter, TIMESTAMP ‘…’, ’TIMESTAMP ‘…’)
> > is
> > > the problematic pattern, but you have tested TIMESTAMP_DIFF(TIMESTAMP
> > ‘…’,
> > > ’TIMESTAMP ‘…’, quarter).
> > > >
> > > > > On Jan 26, 2023, at 9:13 AM, Tanner Clary  > .INVALID>
> > > wrote:
> > > > >
> > > > > Yes I will be putting up a PR with a fix in the next hour or so.
> > Sorry
> > > > > about the hold up!
> > > > >
> > > > > Tanner
> > > > >
> > > > > On Thu, Jan 26, 2023 at 8:09 AM Julian Hyde 
> > > wrote:
> > > > >
> > > > >> I agree that this is a blocker.
> > > > >>
> > > > >> Sergey’s PR illustrates the problem but isn’t a great fix. The fix
> > > should
> > > > >> look for “flag” literals in order to distinguish between the
> > function
> > > > >> variants. The test case should be in a Quidem test rather than
> > > > >> RexBuikderTest.
> > > > >>
> > > > >> Tanner, do you have time to work on a revised PR? In any case I’ll
> > > merge a
> > > > >> fix in the next 8 hours.
> > > > >>
> > > > >> Julian
> > > > >>
> > > > >>> On Jan 26, 2023, at 12:09 AM, Ruben Q L  wrote:
> > > > >>>
> > > > >>> According to our Jira dashboard [1] it seems there's still one
> > > blocker
> > > > >>> issue for 1.33 [2] (it was categorized as blocker because it was a
> > > > >>> regression).
> > > > >>> I guess we should complete this ticket before starting the RC.
> > > > >>>
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > > > >>
> > >
> > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > > >>> [2] https://issues.apache.org/jira/browse/CALCITE-5489
> > > > >>>
> > > >  On Thu, Jan 26, 2023 at 3:44 AM Jess Balint 
> > > wrote:
> > > > 
> > > >  Great, I'll start the build. Let's hold off on further commits
> > > please.
> > > > 
> > > > > On Wed, Jan 25, 2023 at 6:48 PM Julian Hyde 
> > > wrote:
> > > > >
> > > > > I have merged it:
> > > > >
> > > > >
> > > > 
> > > > >>
> > >
> > https://github.com/apache/calcite/commit/20014b6c5b9b57d29248206f19d63ef50f7a5c0f
> > > > >
> > > > > On Wed, Jan 25, 2023 at 2:03 PM Jess Balint 
> > > wrote:
> > > > >>
> > > > >> I'll merge it and then start the rc build.
> > > > >>
> > > > >> On Wed, Jan 25, 2023, 13:04 Julian Hyde 
> > wrote:
> > > > >>
> > > > >>> Oops, I have one more PR ready to merge [1]. If the main branch
> > > is
> > > >  not
> > > > >>> yet closed, let me know, Jess.
> > > > >>>
> > > > >>> Julian
> > > > >>>
> > > > >>> [1] https://issues.apache.org/jira/browse/CALCITE-5283
> > > > >>>
> > > > >>> On Wed, Jan 25, 2023 at 8:14 AM Julian Hyde <
> > > jhyde.apa...@gmail.com>
> > > > >>> wrote:
> > > > 
> > > >  I agree. I merged some PRs last night (before I saw this
> > email)
> > > >  but I
> > > > >>> agree it’s time for an RC. I’ll stop pushing to main. Have at
> > it,
> > > >  Jess!
> > > > 
> > > >  Julian
> > > > 
> > > > > On Jan 24, 2023, at 2:03 PM, Stamatis Zampetakis <
> > > > > zabe...@gmail.com>
> > > > >>> wrote:
> > > > >
> > > > > Thanks for staying 

Re: [DISCUSS] Towards Calcite 1.33.0

[Alessandro Solimando]

Hello,
I think that Julian was referring to CALCITE-5501
 for the intermittent
CI issue, a PR fixing it is now available.

I have no problem waiting for the release to be rolled out before merging
it, just wanted to reach out to see if you prefer to merge it right away,
in case please let me know.

Btw, thanks Jess for being the release manager!

Best regards,
Alessandro

On Fri, 27 Jan 2023 at 23:09, Jess Balint  wrote:

> Thanks, will get the release notes PR up shortly.
>
> On Thu, Jan 26, 2023 at 7:29 PM Julian Hyde  wrote:
>
> > I have merged a fix for CALCITE-5489. Thanks to Tanner and TJ for their
> > help.
> >
> > There is one failure in the build, but I'm pretty sure it is
> > intermittent and is related to CI infrastructure, not our code.
> >
> > Jess, Go ahead and make that RC.
> >
> > Julian
> >
> >
> > On Thu, Jan 26, 2023 at 10:50 AM Julian Hyde 
> > wrote:
> > >
> > > I tried your PR https://github.com/apache/calcite/pull/3048/files. It
> > fixes the bug, but the test cases you have included to not reproduce the
> > bug. I think that TIMESTAMP_DIFF(quarter, TIMESTAMP ‘…’, ’TIMESTAMP ‘…’)
> is
> > the problematic pattern, but you have tested TIMESTAMP_DIFF(TIMESTAMP
> ‘…’,
> > ’TIMESTAMP ‘…’, quarter).
> > >
> > > > On Jan 26, 2023, at 9:13 AM, Tanner Clary  .INVALID>
> > wrote:
> > > >
> > > > Yes I will be putting up a PR with a fix in the next hour or so.
> Sorry
> > > > about the hold up!
> > > >
> > > > Tanner
> > > >
> > > > On Thu, Jan 26, 2023 at 8:09 AM Julian Hyde 
> > wrote:
> > > >
> > > >> I agree that this is a blocker.
> > > >>
> > > >> Sergey’s PR illustrates the problem but isn’t a great fix. The fix
> > should
> > > >> look for “flag” literals in order to distinguish between the
> function
> > > >> variants. The test case should be in a Quidem test rather than
> > > >> RexBuikderTest.
> > > >>
> > > >> Tanner, do you have time to work on a revised PR? In any case I’ll
> > merge a
> > > >> fix in the next 8 hours.
> > > >>
> > > >> Julian
> > > >>
> > > >>> On Jan 26, 2023, at 12:09 AM, Ruben Q L  wrote:
> > > >>>
> > > >>> According to our Jira dashboard [1] it seems there's still one
> > blocker
> > > >>> issue for 1.33 [2] (it was categorized as blocker because it was a
> > > >>> regression).
> > > >>> I guess we should complete this ticket before starting the RC.
> > > >>>
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > >>> [2] https://issues.apache.org/jira/browse/CALCITE-5489
> > > >>>
> > >  On Thu, Jan 26, 2023 at 3:44 AM Jess Balint 
> > wrote:
> > > 
> > >  Great, I'll start the build. Let's hold off on further commits
> > please.
> > > 
> > > > On Wed, Jan 25, 2023 at 6:48 PM Julian Hyde 
> > wrote:
> > > >
> > > > I have merged it:
> > > >
> > > >
> > > 
> > > >>
> >
> https://github.com/apache/calcite/commit/20014b6c5b9b57d29248206f19d63ef50f7a5c0f
> > > >
> > > > On Wed, Jan 25, 2023 at 2:03 PM Jess Balint 
> > wrote:
> > > >>
> > > >> I'll merge it and then start the rc build.
> > > >>
> > > >> On Wed, Jan 25, 2023, 13:04 Julian Hyde 
> wrote:
> > > >>
> > > >>> Oops, I have one more PR ready to merge [1]. If the main branch
> > is
> > >  not
> > > >>> yet closed, let me know, Jess.
> > > >>>
> > > >>> Julian
> > > >>>
> > > >>> [1] https://issues.apache.org/jira/browse/CALCITE-5283
> > > >>>
> > > >>> On Wed, Jan 25, 2023 at 8:14 AM Julian Hyde <
> > jhyde.apa...@gmail.com>
> > > >>> wrote:
> > > 
> > >  I agree. I merged some PRs last night (before I saw this
> email)
> > >  but I
> > > >>> agree it’s time for an RC. I’ll stop pushing to main. Have at
> it,
> > >  Jess!
> > > 
> > >  Julian
> > > 
> > > > On Jan 24, 2023, at 2:03 PM, Stamatis Zampetakis <
> > > > zabe...@gmail.com>
> > > >>> wrote:
> > > >
> > > > Thanks for staying on top of this Jess!
> > > >
> > > > Apart from the two JIRAs marked as blockers (regressions from
> > > > 1.32.0) I
> > > > don't think we should wait much more for getting the release
> > out.
> > > >
> > > > Best,
> > > > Stamatis
> > > >
> > > >> On Tue, Jan 24, 2023 at 10:59 PM Jess Balint <
> > jbal...@gmail.com
> > > >
> > > >>> wrote:
> > > >>
> > > >> The Avatica release is out. I've done what I can on
> > outstanding
> > > >>> reviews:
> > > >>
> > > >>
> > > >>>
> > > >
> > > 
> > > >>
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > > >>
> > > >> Do we have any final requests for PRs to be merged or
> should I
> > > > create
> > > >>> the
> > > >> rc build?
> > > >>
> > > >>> On Sun, Jan 15, 

Re: [DISCUSS] Towards Calcite 1.33.0

[Jess Balint]

Thanks, will get the release notes PR up shortly.

On Thu, Jan 26, 2023 at 7:29 PM Julian Hyde  wrote:

> I have merged a fix for CALCITE-5489. Thanks to Tanner and TJ for their
> help.
>
> There is one failure in the build, but I'm pretty sure it is
> intermittent and is related to CI infrastructure, not our code.
>
> Jess, Go ahead and make that RC.
>
> Julian
>
>
> On Thu, Jan 26, 2023 at 10:50 AM Julian Hyde 
> wrote:
> >
> > I tried your PR https://github.com/apache/calcite/pull/3048/files. It
> fixes the bug, but the test cases you have included to not reproduce the
> bug. I think that TIMESTAMP_DIFF(quarter, TIMESTAMP ‘…’, ’TIMESTAMP ‘…’) is
> the problematic pattern, but you have tested TIMESTAMP_DIFF(TIMESTAMP ‘…’,
> ’TIMESTAMP ‘…’, quarter).
> >
> > > On Jan 26, 2023, at 9:13 AM, Tanner Clary 
> wrote:
> > >
> > > Yes I will be putting up a PR with a fix in the next hour or so. Sorry
> > > about the hold up!
> > >
> > > Tanner
> > >
> > > On Thu, Jan 26, 2023 at 8:09 AM Julian Hyde 
> wrote:
> > >
> > >> I agree that this is a blocker.
> > >>
> > >> Sergey’s PR illustrates the problem but isn’t a great fix. The fix
> should
> > >> look for “flag” literals in order to distinguish between the function
> > >> variants. The test case should be in a Quidem test rather than
> > >> RexBuikderTest.
> > >>
> > >> Tanner, do you have time to work on a revised PR? In any case I’ll
> merge a
> > >> fix in the next 8 hours.
> > >>
> > >> Julian
> > >>
> > >>> On Jan 26, 2023, at 12:09 AM, Ruben Q L  wrote:
> > >>>
> > >>> According to our Jira dashboard [1] it seems there's still one
> blocker
> > >>> issue for 1.33 [2] (it was categorized as blocker because it was a
> > >>> regression).
> > >>> I guess we should complete this ticket before starting the RC.
> > >>>
> > >>>
> > >>> [1]
> > >>>
> > >>
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > >>> [2] https://issues.apache.org/jira/browse/CALCITE-5489
> > >>>
> >  On Thu, Jan 26, 2023 at 3:44 AM Jess Balint 
> wrote:
> > 
> >  Great, I'll start the build. Let's hold off on further commits
> please.
> > 
> > > On Wed, Jan 25, 2023 at 6:48 PM Julian Hyde 
> wrote:
> > >
> > > I have merged it:
> > >
> > >
> > 
> > >>
> https://github.com/apache/calcite/commit/20014b6c5b9b57d29248206f19d63ef50f7a5c0f
> > >
> > > On Wed, Jan 25, 2023 at 2:03 PM Jess Balint 
> wrote:
> > >>
> > >> I'll merge it and then start the rc build.
> > >>
> > >> On Wed, Jan 25, 2023, 13:04 Julian Hyde  wrote:
> > >>
> > >>> Oops, I have one more PR ready to merge [1]. If the main branch
> is
> >  not
> > >>> yet closed, let me know, Jess.
> > >>>
> > >>> Julian
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/CALCITE-5283
> > >>>
> > >>> On Wed, Jan 25, 2023 at 8:14 AM Julian Hyde <
> jhyde.apa...@gmail.com>
> > >>> wrote:
> > 
> >  I agree. I merged some PRs last night (before I saw this email)
> >  but I
> > >>> agree it’s time for an RC. I’ll stop pushing to main. Have at it,
> >  Jess!
> > 
> >  Julian
> > 
> > > On Jan 24, 2023, at 2:03 PM, Stamatis Zampetakis <
> > > zabe...@gmail.com>
> > >>> wrote:
> > >
> > > Thanks for staying on top of this Jess!
> > >
> > > Apart from the two JIRAs marked as blockers (regressions from
> > > 1.32.0) I
> > > don't think we should wait much more for getting the release
> out.
> > >
> > > Best,
> > > Stamatis
> > >
> > >> On Tue, Jan 24, 2023 at 10:59 PM Jess Balint <
> jbal...@gmail.com
> > >
> > >>> wrote:
> > >>
> > >> The Avatica release is out. I've done what I can on
> outstanding
> > >>> reviews:
> > >>
> > >>
> > >>>
> > >
> > 
> > >>
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950
> > >>
> > >> Do we have any final requests for PRs to be merged or should I
> > > create
> > >>> the
> > >> rc build?
> > >>
> > >>> On Sun, Jan 15, 2023 at 2:20 PM Julian Hyde <
> jh...@apache.org>
> > >>> wrote:
> > >>>
> > >>> I have created a jira case to track this release:
> > >>> https://issues.apache.org/jira/browse/CALCITE-5481
> > >>>
> > >>> Also, I am proposing to make a quick Avatica 1.23 release. If
> >  the
> > >>> plans
> > >>> work out, it will be ready for inclusion in Calcite 1.33 from
> > >>> Thursday.
> > >>>
> > >>> Julian
> > >>>
> > >>>
> > >>> On 2023/01/05 20:21:03 Julian Hyde wrote:
> >  Good idea. I have started reviewing those two cases.
> > 
> >  Reviewers are still needed for the other cases.
> > 
> >  Julian
> > 
> > 

[jira] [Created] (CALCITE-5504) Array literals are unparsed incorrectly for the spark dialect

[Jira]

Guillaume Massé created CALCITE-5504:


 Summary: Array literals are unparsed incorrectly for the spark 
dialect
 Key: CALCITE-5504
 URL: https://issues.apache.org/jira/browse/CALCITE-5504
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.32.0
Reporter: Guillaume Massé


They should be in the form: array(1, 2, 3)
However, they are unparsed as ARRAY[1, 2, 3]
 
I would be happy to submit a fix,
Guillaume



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [SUPPORT] Semantics change when applying ConverterRule in CheapestPlanReplacer

[Moritz Mack]

Julian, I’ve created CALCITE-5503 as suggested.

Would you be able to add me as a contributor in Jira and assign the ticket to 
me.
I opened a PR with the test case you’ve asked for and a rather trivial fix.
https://github.com/apache/calcite/pull/3050

Thanks,
Moritz

On 27.01.23, 10:00, "Moritz Mack"  wrote:

Thanks a lot, Julian.
I’ll look into that the next few days.

/Moritz

On 25.01.23, 17:04, "Julian Hyde"  wrote:

It would seem a reasonable and useful enhancement if CheapestPlanReplacer did 
some memoization. If it sees the same RelNode twice it should emit the same 
result. I would be conservative, and memoize based on identity of the RelNodes, 
not just equality.

It would be useful if you could come up with a test case in pure Calcite, then 
log a jira case.

Julian

> On Jan 25, 2023, at 12:56 AM, Moritz Mack  wrote:
>
> Dear Calcite Devs,
>
> I’m currently looking into an issue in the SQL extension [1] of Apache Beam 
> and was hoping to find some advice here.
> Using a bunch of Calcite ConverterRules [2], we convert a RelNode tree into a 
> tree of BeamRelNodes which is then used to build a Beam DAG. Pretty standard 
> I suppose …
>
> What I’m scratching my head about is that applying the converter rules 
> changes the semantics of the graph, which it shouldn’t I thought. Or is that 
> a wrong expectation?
> Here’s a very simple SQL example to illustrate this (see also 
> BeamUnionRelTest [3]):
>
> SELECT  order_id, site_id, price FROM ORDER_DETAILS
> UNION ALL
> SELECT  order_id, site_id, price FROM ORDER_DETAILS
>
> This is where we start at, the corresponding RelNode:
>
> LogicalUnion.NONE(input#0=LogicalProject#8,input#1=LogicalProject#10,all=true)
>
> Once the corresponding conversion rule [4] is applied to above by the 
> CheapestPlanReplacer, but before visiting its old inputs, we get the 
> following:
>
> BeamUnionRel.BEAM_LOGICAL(input#0=RelSubset#25,input#1=RelSubset#25,all=true)
>
> At this point both inputs refer to the same node (#25). However, once 
> visiting the inputs [5] in CheapestPlanReplacer, that semantic information is 
> lost as RelNodes get copied if inputs change [5].
> In below result, the two inputs refer to different nodes:
>
> BeamUnionRel.BEAM_LOGICAL(input#0=BeamCalcRel#49,input#1=BeamCalcRel#50,all=true)
>
> This, however, currently prevents caching of intermediate results in the Beam 
> DAG when this might be beneficial.
>
> Would you have any advice how to better approach this?
> Of course, I could use a stateful copy operation to handle such repeated copy 
> operations with the same parameters. But this seems wrong to be honest.
>
> Thanks a million!
> Kind regards,
> Moritz
>
>
> [1] 
> https://urldefense.com/v3/__https://github.com/apache/beam/issues/24314__;!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrH1HemJw$
> [2] 
> https://urldefense.com/v3/__https://github.com/apache/beam/blob/6ba647333c4c69fb6dfc65929456c7c11570382f/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/planner/BeamRuleSets.java*L136-L152__;Iw!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9Ubryge2rYo$
> [3] 
> https://urldefense.com/v3/__https://github.com/apache/beam/blob/6ba647333c4c69fb6dfc65929456c7c11570382f/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnionRelTest.java*L52-L71__;Iw!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrYLpRpco$
> [4] 
> https://urldefense.com/v3/__https://github.com/apache/beam/blob/a96afe2c57c45a869a622086eaa4f81305f06e72/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/BeamUnionRule.java__;!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrQdhrgjw$
> [5] 
> 

[jira] [Created] (CALCITE-5503) Memoize visited nodes in CheapestPlanReplacer

[Moritz Mack (Jira)]

Moritz Mack created CALCITE-5503:


 Summary: Memoize visited nodes in CheapestPlanReplacer
 Key: CALCITE-5503
 URL: https://issues.apache.org/jira/browse/CALCITE-5503
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Moritz Mack


When using CheapestPlanReplacer, semantics of a RelNode tree change if it 
contains the same node multiple times in case this node has inputs itself that 
have to be replaced. During replacement such nodes get copied on each 
occurrence.

Instead CheapestPlanReplacer should memoize previously visited nodes and emit 
the same result again in the that case.

The test case below illustrates the issue and fails on the current main branch.

 
{code:java}
@Test void testMemoizeInputRelNodes() {
  VolcanoPlanner planner = new VolcanoPlanner();
  planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
  RelOptCluster cluster = newCluster(planner);

  // The rule that triggers the assert rule
  planner.addRule(PhysLeafRule.INSTANCE);
  planner.addRule(GoodSingleRule.INSTANCE);

  // Leaf RelNode
  NoneLeafRel leafRel = new NoneLeafRel(cluster, "a");
  RelNode leafPhy = planner
  .changeTraits(leafRel, cluster.traitSetOf(PHYS_CALLING_CONVENTION));

  // RelNode with leaf RelNode as single input
  NoneSingleRel singleRel = new NoneSingleRel(cluster, leafPhy);
  RelNode singlePhy = planner
  .changeTraits(singleRel, cluster.traitSetOf(PHYS_CALLING_CONVENTION));

  // Binary RelNode with identical input on either side
  PhysBiRel parent = new PhysBiRel(
  cluster, cluster.traitSetOf(PHYS_CALLING_CONVENTION), singlePhy, 
singlePhy);
  planner.setRoot(parent);

  RelNode result = planner.chooseDelegate().findBestExp();

  // Expect inputs to remain identical
  assertEquals(result.getInput(0), result.getInput(1));
} {code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Fwd: several JavaCC warnings "Choice conflict involving two expansions"

[Alessandro Solimando]

Hello everyone,
while checking CI logs I have noticed that we have lots of JavaCC warnings
related to ambiguous prefixes in the productions of one of our grammars.

They also seem related to time functions, for which I have seen several
related developments for BigQuery lately.

Have we verified that our grammar is still behaving properly under this
situation? Have we considered increasing the lookahead value as suggested?
Shall we open a Jira ticket to have a closer look?

Here is an example of CI logs showing the problem (although it is
reproducible locally):
https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull


In what follows the extract that is relevant to the discussion at hand:

> > Task :core:javaCCMain
> Java Compiler Compiler Version 4.0 (Parser Generator)
> (type "javacc" with no arguments for help)
> Reading from file
> /home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
> . . .
> Warning: Output directory
> "/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
> does not exist. Creating the directory.
> Note: UNICODE_INPUT option is specified. Please make sure you create the
> parser/lexer using a Reader with the correct character encoding.
> Warning: Choice conflict involving two expansions at
>  line 4930, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "MICROSECOND"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4931, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "MILLISECOND"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4936, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "DOW"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4937, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "DOY"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4938, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "ISODOW"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4939, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "ISOYEAR"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4940, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "WEEK"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4950, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "QUARTER"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4952, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "EPOCH"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4953, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "DECADE"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4954, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "CENTURY"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 4955, column 5 and line 4956, column 5 respectively.
>  A common prefix is: "MILLENNIUM"
>  Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
>  line 6549, column 9 and line 6551, column 9 respectively.
>  A common prefix is: "WEEK" "("
>  Consider using a lookahead of 3 or more for earlier expansion.
> File "TokenMgrError.java" does not exist.  Will create one.
> File "ParseException.java" does not exist.  Will create one.
> File "Token.java" does not exist.  Will create one.
> File "SimpleCharStream.java" does not exist.  Will create one.
> Parser generated with 0 errors and 14 warnings.


Best regards,
Alessandro

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Michael Mior]

Congratulations and welcome Benchao!

--
Michael Mior
mm...@apache.org


On Fri, Jan 27, 2023 at 4:51 AM Stamatis Zampetakis 
wrote:

> I am pleased to announce that Benchao has accepted an invitation to join
> the Calcite PMC. Benchao has been a consistent and helpful figure in
> the Calcite community for which we are very grateful. We look forward to
> the continued contributions and support.
>
> Please join me in congratulating Benchao!
>
> Stamatis (on behalf of the Calcite PMC)
>

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Ruben Q L]

Congratulations Benchao!
You have become a very important and valuable figure in the community.
Thanks for your help and your contributions!

Best,
Ruben


On Fri, Jan 27, 2023 at 10:29 AM Sergey Nuyanzin 
wrote:

> Congratulations, Benchao!
>
> On Fri, Jan 27, 2023 at 11:27 AM Francis Chuang 
> wrote:
>
> > Congrats, Benchao!
> >
> > On 27/01/2023 9:26 pm, Alessandro Solimando wrote:
> > > Congratulations Benchao, very well deserved!
> > >
> > > Best regards,
> > > Alessandro
> > >
> > > On Fri, 27 Jan 2023 at 10:51, Stamatis Zampetakis 
> > wrote:
> > >
> > >> I am pleased to announce that Benchao has accepted an invitation to
> join
> > >> the Calcite PMC. Benchao has been a consistent and helpful figure in
> > >> the Calcite community for which we are very grateful. We look forward
> to
> > >> the continued contributions and support.
> > >>
> > >> Please join me in congratulating Benchao!
> > >>
> > >> Stamatis (on behalf of the Calcite PMC)
> > >>
> > >
> >
>
>
> --
> Best regards,
> Sergey
>

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Sergey Nuyanzin]

Congratulations, Benchao!

On Fri, Jan 27, 2023 at 11:27 AM Francis Chuang 
wrote:

> Congrats, Benchao!
>
> On 27/01/2023 9:26 pm, Alessandro Solimando wrote:
> > Congratulations Benchao, very well deserved!
> >
> > Best regards,
> > Alessandro
> >
> > On Fri, 27 Jan 2023 at 10:51, Stamatis Zampetakis 
> wrote:
> >
> >> I am pleased to announce that Benchao has accepted an invitation to join
> >> the Calcite PMC. Benchao has been a consistent and helpful figure in
> >> the Calcite community for which we are very grateful. We look forward to
> >> the continued contributions and support.
> >>
> >> Please join me in congratulating Benchao!
> >>
> >> Stamatis (on behalf of the Calcite PMC)
> >>
> >
>


-- 
Best regards,
Sergey

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Francis Chuang]

Congrats, Benchao!

On 27/01/2023 9:26 pm, Alessandro Solimando wrote:

Congratulations Benchao, very well deserved!

Best regards,
Alessandro

On Fri, 27 Jan 2023 at 10:51, Stamatis Zampetakis  wrote:


I am pleased to announce that Benchao has accepted an invitation to join
the Calcite PMC. Benchao has been a consistent and helpful figure in
the Calcite community for which we are very grateful. We look forward to
the continued contributions and support.

Please join me in congratulating Benchao!

Stamatis (on behalf of the Calcite PMC)

Re: [ANNOUNCE] Benchao Li joins Calcite PMC

[Alessandro Solimando]

Congratulations Benchao, very well deserved!

Best regards,
Alessandro

On Fri, 27 Jan 2023 at 10:51, Stamatis Zampetakis  wrote:

> I am pleased to announce that Benchao has accepted an invitation to join
> the Calcite PMC. Benchao has been a consistent and helpful figure in
> the Calcite community for which we are very grateful. We look forward to
> the continued contributions and support.
>
> Please join me in congratulating Benchao!
>
> Stamatis (on behalf of the Calcite PMC)
>

[ANNOUNCE] Benchao Li joins Calcite PMC

[Stamatis Zampetakis]

I am pleased to announce that Benchao has accepted an invitation to join
the Calcite PMC. Benchao has been a consistent and helpful figure in
the Calcite community for which we are very grateful. We look forward to
the continued contributions and support.

Please join me in congratulating Benchao!

Stamatis (on behalf of the Calcite PMC)

Re: [SUPPORT] Semantics change when applying ConverterRule in CheapestPlanReplacer

[Moritz Mack]

Thanks a lot, Julian.
I’ll look into that the next few days.

/Moritz

On 25.01.23, 17:04, "Julian Hyde"  wrote:

It would seem a reasonable and useful enhancement if CheapestPlanReplacer did 
some memoization. If it sees the same RelNode twice it should emit the same 
result. I would be conservative, and memoize based on identity of the RelNodes, 
not just equality.

It would be useful if you could come up with a test case in pure Calcite, then 
log a jira case.

Julian

> On Jan 25, 2023, at 12:56 AM, Moritz Mack  wrote:
>
> Dear Calcite Devs,
>
> I’m currently looking into an issue in the SQL extension [1] of Apache Beam 
> and was hoping to find some advice here.
> Using a bunch of Calcite ConverterRules [2], we convert a RelNode tree into a 
> tree of BeamRelNodes which is then used to build a Beam DAG. Pretty standard 
> I suppose …
>
> What I’m scratching my head about is that applying the converter rules 
> changes the semantics of the graph, which it shouldn’t I thought. Or is that 
> a wrong expectation?
> Here’s a very simple SQL example to illustrate this (see also 
> BeamUnionRelTest [3]):
>
> SELECT  order_id, site_id, price FROM ORDER_DETAILS
> UNION ALL
> SELECT  order_id, site_id, price FROM ORDER_DETAILS
>
> This is where we start at, the corresponding RelNode:
>
> LogicalUnion.NONE(input#0=LogicalProject#8,input#1=LogicalProject#10,all=true)
>
> Once the corresponding conversion rule [4] is applied to above by the 
> CheapestPlanReplacer, but before visiting its old inputs, we get the 
> following:
>
> BeamUnionRel.BEAM_LOGICAL(input#0=RelSubset#25,input#1=RelSubset#25,all=true)
>
> At this point both inputs refer to the same node (#25). However, once 
> visiting the inputs [5] in CheapestPlanReplacer, that semantic information is 
> lost as RelNodes get copied if inputs change [5].
> In below result, the two inputs refer to different nodes:
>
> BeamUnionRel.BEAM_LOGICAL(input#0=BeamCalcRel#49,input#1=BeamCalcRel#50,all=true)
>
> This, however, currently prevents caching of intermediate results in the Beam 
> DAG when this might be beneficial.
>
> Would you have any advice how to better approach this?
> Of course, I could use a stateful copy operation to handle such repeated copy 
> operations with the same parameters. But this seems wrong to be honest.
>
> Thanks a million!
> Kind regards,
> Moritz
>
>
> [1] 
> https://urldefense.com/v3/__https://github.com/apache/beam/issues/24314__;!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrH1HemJw$
> [2] 
> https://urldefense.com/v3/__https://github.com/apache/beam/blob/6ba647333c4c69fb6dfc65929456c7c11570382f/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/planner/BeamRuleSets.java*L136-L152__;Iw!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9Ubryge2rYo$
> [3] 
> https://urldefense.com/v3/__https://github.com/apache/beam/blob/6ba647333c4c69fb6dfc65929456c7c11570382f/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnionRelTest.java*L52-L71__;Iw!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrYLpRpco$
> [4] 
> https://urldefense.com/v3/__https://github.com/apache/beam/blob/a96afe2c57c45a869a622086eaa4f81305f06e72/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/BeamUnionRule.java__;!!CiXD_PY!UU9RIn-JXhMMhXinYi6Iq3oHsLnmlLXkqWMc4A6J85FY83_atUeh4tseWCHR_L5B7bcBk_fj9UbrQdhrgjw$
> [5] 
>