date:20211014

Re: [DISCUSS] Next releases

2021-10-14 Thread Julian Hyde

I'm ready to produce the first release candidate, but I am blocked by
a Gradle issue. Vladimir, I'd be grateful if you could take a look.

https://issues.apache.org/jira/browse/CALCITE-4853

Julian

On Thu, Oct 14, 2021 at 10:37 AM Julian Hyde  wrote:
>
> I have written a first draft of the release notes and opened a pull
> request. Can people please review the release notes and check
> contributor names.
>
> https://github.com/julianhyde/calcite/blob/4835-release-1.28/site/_docs/history.md
>
> https://github.com/apache/calcite/pull/2584
>
> Master branch is still open for commits, but please be careful not to
> break the build. I'll probably start a vote today or tomorrow.
>
> On Sat, Oct 9, 2021 at 11:24 PM Julian Hyde  wrote:
> >
> > Now that Avatica’s release 1.19 is final, the master branch is open for 
> > commits. I’ll make the official release announcement on Monday.
> >
> > I’m targeting Wednesday for the first release candidate for Calcite version 
> > 1.28.
> >
> > Julian
> >
> >
> > > On Oct 7, 2021, at 12:32 PM, Jacques Nadeau  wrote:
> > >
> > > I merged the last patch and have linked the release ticket to a release
> > > note gist for the changes. Thanks.
> > >
> > > On Wed, Oct 6, 2021, 10:21 PM Jacques Nadeau  wrote:
> > >
> > >> FYI, the last patch for immutables is up and passing all checks. I'll do
> > >> another pass in the morning to make sure everything looks good and then
> > >> will merge if I don't hear any concerns.
> > >>
> > >> https://github.com/apache/calcite/pull/2568
> > >>
> > >>
> > >> On Wed, Oct 6, 2021, 1:04 PM Jacques Nadeau  wrote:
> > >>
> > >>> Definitely. I'm also trying to add helpful comments in a couple of key
> > >>> javadoc locations.
> > >>>
> > >>> On Wed, Oct 6, 2021 at 1:02 PM Julian Hyde 
> > >>> wrote:
> > >>>
> >  Thanks for following through on this large change, Jacques. Be sure to
> >  document the changes people will need to make if they have defined 
> >  their
> >  own rules.
> > 
> > 
> > > On Oct 6, 2021, at 12:59 PM, Jacques Nadeau 
> >  wrote:
> > >
> > > I'm hoping (fingers crossed) to have the final Immutables patch up
> >  today
> > > and if all goes well, get that merged before the release.
> > >
> > > On Wed, Oct 6, 2021 at 11:34 AM Julian Hyde  wrote:
> > >
> > >> So, the vote for Avatica 1.19 is underway, and we will likely have a
> > >> release in 4 or 5 days.
> > >>
> > >> Next up, Calcite release 1.28. I'll be release manager for that too,
> >  and
> > >> I'd like to start the vote in a week. I have three asks:
> > >>
> > >> 1. Start stabilizing the build. If you have a change that might cause
> > >> build instability, ask on the dev list before you commit it. The
> >  recent
> > >> changes for Gradle, JDK 16/17 and Immutables have been destabilizing,
> >  so I
> > >> ask those authors
> > >>
> > >> 2. I have logged https://issues.apache.org/jira/browse/CALCITE-4835.
> >  If
> > >> you have made breaking changes since 1.27, please document them in
> >  that
> > >> case, and I will add them to the release notes. If there are bugs
> >  that MUST
> > >> be fixed in 1.28, link to them from that case.
> > >>
> > >> 3. Can I have a few volunteers to go through the PRs and merge any
> >  that
> > >> are ready? (Please reply to this email, so we know you are working on
> >  it.)
> > >>
> > >> Julian
> > >>
> > >>
> > >> On 2021/09/27 19:21:51, Alessandro Solimando <
> > >> alessandro.solima...@gmail.com> wrote:
> > >>> Hi everyone,
> > >>> I have prepared the PR, you can find it here:
> > >>> https://github.com/apache/calcite-avatica/pull/154.
> > >>>
> > >>> Best regards,
> > >>> Alessandro
> > >>>
> > >>> On Mon, 27 Sept 2021 at 20:27, Alessandro Solimando <
> > >>> alessandro.solima...@gmail.com> wrote:
> > >>>
> >  Hi Julian,
> >  I can help with PR-143
> >  , since I have
> >  contributed to the modified classes in the past.
> > 
> >  I will still need a committer to merge but I can rebase on master
> >  and
> > >> fix
> >  issues, if any.
> > 
> >  Best regards,
> >  Alessandro
> > 
> >  On Mon, 27 Sept 2021 at 07:52, Julian Hyde 
> > >> wrote:
> > 
> > > Can people please try to review and merge the Avatica PRs this
> >  week?
> > > Several look ready:
> > >
> > > https://github.com/apache/calcite-avatica/pulls
> > >
> > > Julian
> > >
> > >
> > >
> > >> On Sep 24, 2021, at 6:22 PM, Haisheng Yuan 
> > >> wrote:
> > >>
> > >> Sounds good!
> > >>
> > >> On 2021/09/25 01:13:26, Julian Hyde  wrote:
> > >>> I propose to

[jira] [Created] (CALCITE-4854) SubstitutionVisitor exec aggregate-match fail, which query with grouping literal alone

2021-10-14 Thread Xurenhe (Jira)

Xurenhe created CALCITE-4854:


 Summary: SubstitutionVisitor exec aggregate-match fail, which 
query with grouping literal alone
 Key: CALCITE-4854
 URL: https://issues.apache.org/jira/browse/CALCITE-4854
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Xurenhe


This case execute SubstitutionVisitor fail, but it need be mv-match.

 
{code:java}
//org.apache.calcite.test.MaterializedViewSubstitutionVisitorTest#test
@Test void test() {
  final String mv = ""
  + "select \"deptno\", \"empid\", count(*)\n"
  + "from \"emps\"\n"
  + "group by \"deptno\", \"empid\"";
  final String query = ""
  + "select '1', count(*)\n"
  + "from \"emps\"\n"
  + "group by '1'";
  sql(mv, query).ok();
}
{code}
 

 

I found that [CALCITE-4779|https://issues.apache.org/jira/browse/CALCITE-4779] 
have done some pre-handler to canonicalize query's relnode, but it‘s not 
applicable for this test case.

Because, AGGREGATE_PROJECT_PULL_UP_CONSTANTS will stop to transform, if size of 
grouping is one.

So, I try to enhance UnifyRules to support it.
 * AggregateToAggregateUnifyRule
 * AggregateOnCalcToAggregateUnifyRule

 

Hi community, Is my solution correct?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-4853) Gradle could not determine the dependencies of task ':syncPreviewSiteRepo'

2021-10-14 Thread Julian Hyde (Jira)

Julian Hyde created CALCITE-4853:


 Summary: Gradle could not determine the dependencies of task 
':syncPreviewSiteRepo'
 Key: CALCITE-4853
 URL: https://issues.apache.org/jira/browse/CALCITE-4853
 Project: Calcite
  Issue Type: Bug
  Components: build
Reporter: Julian Hyde


I got the following error while following the instructions to [make a Calcite 
release 
candidate|https://calcite.apache.org/docs/howto.html#making-a-release-candidate].

{noformat}
$ ./gradlew prepareVote -Prc=0
Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use 
--status for details

> Configure project :
Building Apache Calcite 1.28.0

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':syncPreviewSiteRepo'.
> Could not resolve all dependencies for configuration ':previewSite'.
   > Querying the mapped value of flatmap(provider(task 'distTar', class 
org.gradle.api.tasks.bundling.Tar)) before task ':release:distTar' has 
completed is not supported

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-4852) RelToSqlConverter creates unparseable SQL string from right associative multi-way join.

2021-10-14 Thread Ian Bertolacci (Jira)

Ian Bertolacci created CALCITE-4852:
---

 Summary: RelToSqlConverter creates unparseable SQL string from 
right associative multi-way join.
 Key: CALCITE-4852
 URL: https://issues.apache.org/jira/browse/CALCITE-4852
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: Ian Bertolacci


As discussed in CALCITE-35, Calcite cannot parse parenthesized join expressions 
(such as `select ... from A join (B join C)`).
But as suggested in CALCITE-2152, those expressions can be converted into 
parenthesized select on the join expression (`select ... from A join (select 
... from B join C)`).

However, RelToSqlConverter will convert the RelNode representation of 
right-associative joins into a parenthesized join expression, which is 
unparseable.

For example, this RelNode tree
{code}
LogicalProject(_T5_ID=[$2], C0_51=[$3], _T1_ID=[$1], C0_53=[$4], _T3_ID=[$0])
  LogicalJoin(condition=[=($0, $4)], joinType=[inner])
LogicalProject(_T3_ID=[$0])
  LogicalTableScan(table=[[QUERY, T3]], fields=[[0, 1, 2]])
  // RHS child of right-associate join
  LogicalJoin(condition=[=($0, $2)], joinType=[inner])
LogicalProject(_T1_ID=[$0])
  LogicalTableScan(table=[[QUERY, T1]], fields=[[0, 1, 2]])
LogicalProject(_T5_ID=[$0], C0_51=[$1], C0_53=[$3])
  LogicalTableScan(table=[[QUERY, T5]], fields=[[0, 1, 2, 3]])
{code}
creates the following unparseable SQL string
{code}
SELECT `t1`.`_t5_id`,
   `t1`.`c0_51`,
   `t0`.`_t1_id`,
   `t1`.`c0_53`,
   `t`.`_t3_id`
FROM   (SELECT `id` AS `_T3_ID`
FROM   `query`.`t3`) AS `t`
   INNER JOIN ((SELECT `id` AS `_T1_ID`
   FROM   `query`.`t1`) AS `t0`
   INNER JOIN (SELECT `id` AS `_T5_ID`,
  `c0_51`,
  `c0_53`
   FROM   `query`.`t5`) AS `t1`
   ON `t0`.`_t1_id` = `t1`.`c0_51`)
   ON `t`.`_t3_id` = `t1`.`c0_53`
{code}

This is an issue, because it is very easy to make such trees, and ideally, all 
SQL strings generated by Calcite would also be parseable by Calcite.

To get around this, we found that the insertion of a projection node between a 
join and it's RHS child (the problematic join) forces RelToSqlConverter to 
create a parenthesized select statement, which *is* parseable, and (at least 
for us) semantically identical.

For example, this virtually identical tree:
{code}
LogicalProject(_T5_ID=[$2], C0_51=[$3], _T1_ID=[$1], C0_53=[$4], _T3_ID=[$0])
  LogicalJoin(condition=[=($0, $4)], joinType=[inner])
LogicalProject(_T3_ID=[$0])
  LogicalTableScan(table=[[QUERY, T3]], fields=[[0, 1, 2]])
// Inserted projection on top of RHS child
LogicalProject(_T1_ID=[$0], _T5_ID=[$1], C0_51=[$2], C0_53=[$3])
  // RHS child of right-associate join
  LogicalJoin(condition=[=($0, $2)], joinType=[inner])
LogicalProject(_T1_ID=[$0])
  LogicalTableScan(table=[[QUERY, T1]], fields=[[0, 1, 2]])
LogicalProject(_T5_ID=[$0], C0_51=[$1], C0_53=[$3])
  LogicalTableScan(table=[[QUERY, T5]], fields=[[0, 1, 2, 3]])
{code}
creates the following parseable SQL string
{code}
SELECT `t2`.`_t5_id`,
   `t2`.`c0_51`,
   `t2`.`_t1_id`,
   `t2`.`c0_53`,
   `t`.`_t3_id`
FROM   (SELECT `id` AS `_T3_ID`
FROM   `query`.`t3`) AS `t`
   INNER JOIN (SELECT *
   FROM   (SELECT `id` AS `_T1_ID`
   FROM   `query`.`t1`) AS `t0`
  INNER JOIN (SELECT `id` AS `_T5_ID`,
 `c0_51`,
 `c0_53`
  FROM   `query`.`t5`) AS `t1`
  ON `t0`.`_t1_id` = `t1`.`c0_51`) AS `t2`
   ON `t`.`_t3_id` = `t2`.`c0_53`
{code}

We solved this our RelToSqlConverter extending class by cloning the parent join 
and inserting a projection between it and an RHS join:
{code}
override def visit(join: Join): Result = {
if (join.getRight.isInstanceOf[Join]) {
  super.visit(
join.copy(
  join.getTraitSet,
  join.getCondition,
  join.getLeft,
  new LogicalProject(
join.getCluster,
join.getTraitSet,
join.getHints,
join.getRight,
// Construct projection of all expressions from the rhs join
join.getRight.getRowType.getFieldList.asScala.map(field => new 
RexInputRef(field.getIndex, field.getType)).asJava,
join.getRight.getRowType
  ),
  join.getJoinType,
  join.isSemiJoinDone
)
  )
} else {
  super.visit(join)
}
  }
{code}
(we tried to do it "correctly" by constructing a proper Result object, but 
found it too difficult. Hopefully

Re: [DISCUSS] Draft board report for Oct 2021

2021-10-14 Thread Julian Hyde

The contributors are the people who have made commits that appear in
this release. You can generate the list as follows:

$ ./sqlsh "select distinct author from git_commits where
commit_timestamp > DATE '2021-06-03' order by 1 "


On Thu, Oct 14, 2021 at 9:57 AM Michael Mior  wrote:
>
> LGTM, although we have new committers to add now.
> --
> Michael Mior
> mm...@apache.org
>
> Le mer. 6 oct. 2021 à 03:15, Haisheng Yuan  a écrit :
> >
> > Attached below is a draft of this month's board report. I plan to submit it 
> > on Oct 12. Please let me know if you have additions or corrections.
> >
> > ## Description:
> > The mission of Calcite is the creation and maintenance of software related 
> > to
> > Dynamic data management framework
> >
> > ## Issues:
> > There are no issues requiring board attention.
> >
> > ## Membership Data:
> > Apache Calcite was founded 2015-10-21 (6 years ago)
> > There are currently 53 committers and 23 PMC members in this project.
> > The Committer-to-PMC ratio is roughly 7:3.
> >
> > Community changes, past quarter:
> > - No new PMC members. Last addition was Ruben Q L on 2020-08-09.
> > - No new committers. Last addition was Vladimir Ozerov on 2021-06-23.
> >
> > ## Project Activity:
> > No new version was released.
> >
> > At ApacheCon 2021, September 22, 2021, Vladimir Ozerov gave a talk on 
> > building
> > modern SQL query optimizers with Apache Calcite.
> >
> > At Strange Loop 2021, September 30, 2021, Julian Hyde gave a talk on Morel,
> > a functional query language.
> >
> > ## Community Health:
> > The overall activity in the community has increased slightly in the past
> > few months, specifically 47% more opened PRs, 25% more closed PRs on GitHub.
> > But the number of active reviewers still remains small.
> >
> > Thanks,
> > Haisheng Yuan

Re: [DISCUSS] Next releases

2021-10-14 Thread Julian Hyde

I have written a first draft of the release notes and opened a pull
request. Can people please review the release notes and check
contributor names.

https://github.com/julianhyde/calcite/blob/4835-release-1.28/site/_docs/history.md

https://github.com/apache/calcite/pull/2584

Master branch is still open for commits, but please be careful not to
break the build. I'll probably start a vote today or tomorrow.

On Sat, Oct 9, 2021 at 11:24 PM Julian Hyde  wrote:
>
> Now that Avatica’s release 1.19 is final, the master branch is open for 
> commits. I’ll make the official release announcement on Monday.
>
> I’m targeting Wednesday for the first release candidate for Calcite version 
> 1.28.
>
> Julian
>
>
> > On Oct 7, 2021, at 12:32 PM, Jacques Nadeau  wrote:
> >
> > I merged the last patch and have linked the release ticket to a release
> > note gist for the changes. Thanks.
> >
> > On Wed, Oct 6, 2021, 10:21 PM Jacques Nadeau  wrote:
> >
> >> FYI, the last patch for immutables is up and passing all checks. I'll do
> >> another pass in the morning to make sure everything looks good and then
> >> will merge if I don't hear any concerns.
> >>
> >> https://github.com/apache/calcite/pull/2568
> >>
> >>
> >> On Wed, Oct 6, 2021, 1:04 PM Jacques Nadeau  wrote:
> >>
> >>> Definitely. I'm also trying to add helpful comments in a couple of key
> >>> javadoc locations.
> >>>
> >>> On Wed, Oct 6, 2021 at 1:02 PM Julian Hyde 
> >>> wrote:
> >>>
>  Thanks for following through on this large change, Jacques. Be sure to
>  document the changes people will need to make if they have defined their
>  own rules.
> 
> 
> > On Oct 6, 2021, at 12:59 PM, Jacques Nadeau 
>  wrote:
> >
> > I'm hoping (fingers crossed) to have the final Immutables patch up
>  today
> > and if all goes well, get that merged before the release.
> >
> > On Wed, Oct 6, 2021 at 11:34 AM Julian Hyde  wrote:
> >
> >> So, the vote for Avatica 1.19 is underway, and we will likely have a
> >> release in 4 or 5 days.
> >>
> >> Next up, Calcite release 1.28. I'll be release manager for that too,
>  and
> >> I'd like to start the vote in a week. I have three asks:
> >>
> >> 1. Start stabilizing the build. If you have a change that might cause
> >> build instability, ask on the dev list before you commit it. The
>  recent
> >> changes for Gradle, JDK 16/17 and Immutables have been destabilizing,
>  so I
> >> ask those authors
> >>
> >> 2. I have logged https://issues.apache.org/jira/browse/CALCITE-4835.
>  If
> >> you have made breaking changes since 1.27, please document them in
>  that
> >> case, and I will add them to the release notes. If there are bugs
>  that MUST
> >> be fixed in 1.28, link to them from that case.
> >>
> >> 3. Can I have a few volunteers to go through the PRs and merge any
>  that
> >> are ready? (Please reply to this email, so we know you are working on
>  it.)
> >>
> >> Julian
> >>
> >>
> >> On 2021/09/27 19:21:51, Alessandro Solimando <
> >> alessandro.solima...@gmail.com> wrote:
> >>> Hi everyone,
> >>> I have prepared the PR, you can find it here:
> >>> https://github.com/apache/calcite-avatica/pull/154.
> >>>
> >>> Best regards,
> >>> Alessandro
> >>>
> >>> On Mon, 27 Sept 2021 at 20:27, Alessandro Solimando <
> >>> alessandro.solima...@gmail.com> wrote:
> >>>
>  Hi Julian,
>  I can help with PR-143
>  , since I have
>  contributed to the modified classes in the past.
> 
>  I will still need a committer to merge but I can rebase on master
>  and
> >> fix
>  issues, if any.
> 
>  Best regards,
>  Alessandro
> 
>  On Mon, 27 Sept 2021 at 07:52, Julian Hyde 
> >> wrote:
> 
> > Can people please try to review and merge the Avatica PRs this
>  week?
> > Several look ready:
> >
> > https://github.com/apache/calcite-avatica/pulls
> >
> > Julian
> >
> >
> >
> >> On Sep 24, 2021, at 6:22 PM, Haisheng Yuan 
> >> wrote:
> >>
> >> Sounds good!
> >>
> >> On 2021/09/25 01:13:26, Julian Hyde  wrote:
> >>> I propose to start a vote for Avatica 1.19 one week from now,
>  and a
> >>> vote for Calcite 1.28 two weeks from now. I'll be RM for both.
>  How
> >>> does that timing sound?
> >>>
> >>> Julian
> >>>
> >>> On Fri, Sep 24, 2021 at 10:12 AM James Starr <
>  jamesst...@gmail.com
> >>>
> > wrote:
> 
>  When is calcite planning on going to 2.0?
> 
>  On Fri, Sep 24, 2021 at 3:35 AM Stamatis Zampetakis <
> >

Re: [DISCUSS] Draft board report for Oct 2021

2021-10-14 Thread Michael Mior

LGTM, although we have new committers to add now.
--
Michael Mior
mm...@apache.org

Le mer. 6 oct. 2021 à 03:15, Haisheng Yuan  a écrit :
>
> Attached below is a draft of this month's board report. I plan to submit it 
> on Oct 12. Please let me know if you have additions or corrections.
>
> ## Description:
> The mission of Calcite is the creation and maintenance of software related to
> Dynamic data management framework
>
> ## Issues:
> There are no issues requiring board attention.
>
> ## Membership Data:
> Apache Calcite was founded 2015-10-21 (6 years ago)
> There are currently 53 committers and 23 PMC members in this project.
> The Committer-to-PMC ratio is roughly 7:3.
>
> Community changes, past quarter:
> - No new PMC members. Last addition was Ruben Q L on 2020-08-09.
> - No new committers. Last addition was Vladimir Ozerov on 2021-06-23.
>
> ## Project Activity:
> No new version was released.
>
> At ApacheCon 2021, September 22, 2021, Vladimir Ozerov gave a talk on building
> modern SQL query optimizers with Apache Calcite.
>
> At Strange Loop 2021, September 30, 2021, Julian Hyde gave a talk on Morel,
> a functional query language.
>
> ## Community Health:
> The overall activity in the community has increased slightly in the past
> few months, specifically 47% more opened PRs, 25% more closed PRs on GitHub.
> But the number of active reviewers still remains small.
>
> Thanks,
> Haisheng Yuan

Re: [ANNOUNCE] New committer: Zhaohui Xu

2021-10-14 Thread Michael Mior

Welcome Zhaohui!
--
Michael Mior
mm...@apache.org

Le mer. 6 oct. 2021 à 16:48, Stamatis Zampetakis  a écrit :
>
> Apache Calcite's Project Management Committee (PMC) has invited Zhaohui Xu
> to
> become a committer, and we are pleased to announce that they have accepted.
>
> Numbers speak for themselves and Zhaohui has over 30 commits already in
> master
> and more than 20 open pull requests waiting to get in. Great record so far
> including
> (but not limited to) improvements and fixes in the view based
> rewriting modules,
> JSON serialization, metadata, and field trimming.
>
> Zhaohui, welcome, thank you for your contributions, and we look forward to
> your
> further interactions with the community! If you wish, please feel free to
> tell
> us more about yourself and what you are working on.
>
> Stamatis (on behalf of the Apache Calcite PMC)

Re: [DISCUSS] Next releases

[jira] [Created] (CALCITE-4854) SubstitutionVisitor exec aggregate-match fail, which query with grouping literal alone

[jira] [Created] (CALCITE-4853) Gradle could not determine the dependencies of task ':syncPreviewSiteRepo'

[jira] [Created] (CALCITE-4852) RelToSqlConverter creates unparseable SQL string from right associative multi-way join.

Re: [DISCUSS] Draft board report for Oct 2021

Re: [DISCUSS] Next releases

Re: [DISCUSS] Draft board report for Oct 2021

Re: [ANNOUNCE] New committer: Zhaohui Xu

8 matches

Site Navigation

Mail list logo

Footer information