Re: Google Summer of Code 2018

[Mark Hammond]

Thanks Julian for the context. I have no insight on Sigma, unfortunately.

I'm working toward adding such functionality to an Elixir Ecto based platform, 
and so will sketch something in due course - just concerned this may not be in 
time for a GSoC submission.

Cheers,
Mark.

> On 28 Feb 2018, at 08:09, Julian Hyde  wrote:
> 
> Mark,
> 
> Do you know if Sigma is built using Calcite? 
> 
> I’m totally in agreement about the power of SQL. You can create 
> access-controlled views (or inject filters based on tenant id), give your 
> end-users access to that layer, and you know that your users will get 
> performance without being able to subvert security. It helps a lot that SQL 
> is mathematically closed. If your users don’t like SQL, layer a non-SQL 
> language on top of the relational algebra if you like.
> 
> I’d appreciate if you could sketch out the JSON feature set that you think 
> Calcite needs. We already have 
> https://issues.apache.org/jira/browse/CALCITE-950 
> ; read 
> https://issues.apache.org/jira/browse/DRILL-6035 
>  for a (sobering) overview 
> of what is needed to fully support reading JSON data.
> 
> Julian
> 
> 
>> On Feb 27, 2018, at 3:25 PM, Mark Hammond  wrote:
>> 
>> I'd like to suggest developing a convenient way to compose WHERE constraints 
>> on base queries, via the JSON api.
>> 
>> Having a convenient way to inject rigorous application tenancy constraints 
>> would be huge boon for SaaS style applications. 
>> 
>> Calcite, with its JSON driver, could effectively enable any application to 
>> safely expose SQL-based queries, and remove the need for limited-capability 
>> domain specific languages. 
>> 
>> Calcite can already restrict queries to whitelisted operators, and the 
>> optimiser can deftly handle a bag of tenancy-related constraints.
>> 
>> It would be possible to extend further by clamping query COST, etc.
>> 
>> E.g. this capability has been productised by stripe, 
>> https://stripe.com/us/sigma.
>> 
>> Cheers,
>> Mark.
>> 
>> 
>>> On 25 Feb 2018, at 08:24, Michael Mior  wrote:
>>> 
>>> You have probably seen that Apache was accepted as an organization for this
>>> year's GSoC. I thought I would see if anyone in the Calcite community can
>>> think of any issues that would be a good fit. It's no guarantee we would
>>> get someone to work on it, but it could be a good push to move some
>>> isolated bits of functionality forward that may not get much attention
>>> otherwise.
>>> 
>>> --
>>> Michael Mior
>>> mm...@apache.org
> 

[jira] [Created] (CALCITE-2196) Update codeGen of metadataHandler in JaninoRelMetadataProvider to include Class definition at begin of dynamic code.

[jingzhang (JIRA)]

jingzhang created CALCITE-2196:
--

 Summary: Update codeGen of metadataHandler in 
JaninoRelMetadataProvider to include Class definition at begin of dynamic code.
 Key: CALCITE-2196
 URL: https://issues.apache.org/jira/browse/CALCITE-2196
 Project: Calcite
  Issue Type: Bug
  Components: core
Reporter: jingzhang
Assignee: Julian Hyde


JaninoRelMetadataProvider uses IClassBodyEvaluator to compile generated code.  
The generated classes can not be debugged in interactively using Idea using the 
method referenced by http://janino-compiler.github.io/janino/ because generated 
code only contains classBody. 
If using ISimpleCompiler to compile complete compile unit instead of classBody, 
we could step into the code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Google Summer of Code 2018

[Aman Sinha]

Good discussion.  Some thoughts in no particular priority order:

   - I think we should also look at the SQL extensions that have been
   proposed in SQL++  [1] .. for instance the WHERE clause can contain a
   QuantifiedExpression such as

  QuantifiedExpression ::= ( (|) |  )
Variable  Expression ( "," Variable "in" Expression )*
  Expression ()?

   - WHERE predicates on JSON fields should be expressed in a manner that
   allows determining selectivity estimates using NDV and Histograms,
   otherwise cost based optimizations would not be possible.
   - Also, what is this community's thoughts on Higher-Order/Lambda
   functions that some products have to operate on JSON arrays and maps etc.
   ?


[1]
https://ci.apache.org/projects/asterixdb/sqlpp/manual.html#Comparison_operators

-Aman

On Tue, Feb 27, 2018 at 4:09 PM, Julian Hyde  wrote:

> Mark,
>
> Do you know if Sigma is built using Calcite?
>
> I’m totally in agreement about the power of SQL. You can create
> access-controlled views (or inject filters based on tenant id), give your
> end-users access to that layer, and you know that your users will get
> performance without being able to subvert security. It helps a lot that SQL
> is mathematically closed. If your users don’t like SQL, layer a non-SQL
> language on top of the relational algebra if you like.
>
> I’d appreciate if you could sketch out the JSON feature set that you think
> Calcite needs. We already have https://issues.apache.org/
> jira/browse/CALCITE-950 ;
> read https://issues.apache.org/jira/browse/DRILL-6035 <
> https://issues.apache.org/jira/browse/DRILL-6035> for a (sobering)
> overview of what is needed to fully support reading JSON data.
>
> Julian
>
>
> > On Feb 27, 2018, at 3:25 PM, Mark Hammond 
> wrote:
> >
> > I'd like to suggest developing a convenient way to compose WHERE
> constraints on base queries, via the JSON api.
> >
> > Having a convenient way to inject rigorous application tenancy
> constraints would be huge boon for SaaS style applications.
> >
> > Calcite, with its JSON driver, could effectively enable any application
> to safely expose SQL-based queries, and remove the need for
> limited-capability domain specific languages.
> >
> > Calcite can already restrict queries to whitelisted operators, and the
> optimiser can deftly handle a bag of tenancy-related constraints.
> >
> > It would be possible to extend further by clamping query COST, etc.
> >
> > E.g. this capability has been productised by stripe,
> https://stripe.com/us/sigma.
> >
> > Cheers,
> > Mark.
> >
> >
> >> On 25 Feb 2018, at 08:24, Michael Mior  wrote:
> >>
> >> You have probably seen that Apache was accepted as an organization for
> this
> >> year's GSoC. I thought I would see if anyone in the Calcite community
> can
> >> think of any issues that would be a good fit. It's no guarantee we would
> >> get someone to work on it, but it could be a good push to move some
> >> isolated bits of functionality forward that may not get much attention
> >> otherwise.
> >>
> >> --
> >> Michael Mior
> >> mm...@apache.org
>
>

Re: [DISCUSS] Towards Avatica 1.11.0

[Laurent Goujon]

I just posted a pull request for
https://issues.apache.org/jira/browse/CALCITE-2140: this is a small
improvement for AvaticaStatement to add compatibility with DataGrip and
other tools. It would be a nice to have for 1.11.0

Laurent

On Tue, Feb 27, 2018 at 2:44 PM, F21  wrote:

> Also would like to see https://issues.apache.org/jira
> /projects/CALCITE/issues/CALCITE-1951 worked on.
>
> Francis
>
> On 28/02/2018 9:34 AM, Julian Hyde wrote:
>
>> OK, new thread just for the Avatica release. What’s left to do before
>> 1.11? What can we accomplish before Monday?
>>
>> There are 20 open JIRA cases tagged for 1.11:
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%
>> 20CALCITE%20AND%20fixVersion%20%3D%20avatica-1.11.0%20and%
>> 20status%20!%3D%20Resolved%20 > a/issues/?jql=project%20=%20CALCITE%20AND%20fixVersion%20=%
>> 20avatica-1.11.0%20and%20status%20!=%20Resolved>.
>>
>> Josh, 12 of those issues are assigned to you. Do you intend to fix any of
>> those? If so, make them dependencies of https://issues.apache.org/jira
>> /browse/CALCITE-2182 > >.
>>
>> Haohui, you have https://issues.apache.org/jira/browse/CALCITE-1884 <
>> https://issues.apache.org/jira/browse/CALCITE-1884>. Please update the
>> case when you make progress.
>>
>> I’ll take a shot at https://issues.apache.org/jira/browse/CALCITE-1928 <
>> https://issues.apache.org/jira/browse/CALCITE-1928>, review
>> https://github.com/apache/calcite-avatica/pull/23 <
>> https://github.com/apache/calcite-avatica/pull/23> =
>> https://issues.apache.org/jira/browse/CALCITE-508 <
>> https://issues.apache.org/jira/browse/CALCITE-508>, commit
>> https://github.com/apache/calcite-avatica/pull/16 <
>> https://github.com/apache/calcite-avatica/pull/16>, and of course write
>> the release notes.
>>
>> Anything else that needs to be done, or any fixes/pull requests that
>> people would like to get in?
>>
>> Julian
>>
>>
>>
>>
>

Re: Google Summer of Code 2018

[Julian Hyde]

Mark,

Do you know if Sigma is built using Calcite? 

I’m totally in agreement about the power of SQL. You can create 
access-controlled views (or inject filters based on tenant id), give your 
end-users access to that layer, and you know that your users will get 
performance without being able to subvert security. It helps a lot that SQL is 
mathematically closed. If your users don’t like SQL, layer a non-SQL language 
on top of the relational algebra if you like.

I’d appreciate if you could sketch out the JSON feature set that you think 
Calcite needs. We already have 
https://issues.apache.org/jira/browse/CALCITE-950 
; read 
https://issues.apache.org/jira/browse/DRILL-6035 
 for a (sobering) overview of 
what is needed to fully support reading JSON data.

Julian


> On Feb 27, 2018, at 3:25 PM, Mark Hammond  wrote:
> 
> I'd like to suggest developing a convenient way to compose WHERE constraints 
> on base queries, via the JSON api.
> 
> Having a convenient way to inject rigorous application tenancy constraints 
> would be huge boon for SaaS style applications. 
> 
> Calcite, with its JSON driver, could effectively enable any application to 
> safely expose SQL-based queries, and remove the need for limited-capability 
> domain specific languages. 
> 
> Calcite can already restrict queries to whitelisted operators, and the 
> optimiser can deftly handle a bag of tenancy-related constraints.
> 
> It would be possible to extend further by clamping query COST, etc.
> 
> E.g. this capability has been productised by stripe, 
> https://stripe.com/us/sigma.
> 
> Cheers,
> Mark.
> 
> 
>> On 25 Feb 2018, at 08:24, Michael Mior  wrote:
>> 
>> You have probably seen that Apache was accepted as an organization for this
>> year's GSoC. I thought I would see if anyone in the Calcite community can
>> think of any issues that would be a good fit. It's no guarantee we would
>> get someone to work on it, but it could be a good push to move some
>> isolated bits of functionality forward that may not get much attention
>> otherwise.
>> 
>> --
>> Michael Mior
>> mm...@apache.org

Re: Google Summer of Code 2018

[Mark Hammond]

I'd like to suggest developing a convenient way to compose WHERE constraints on 
base queries, via the JSON api.

Having a convenient way to inject rigorous application tenancy constraints 
would be huge boon for SaaS style applications. 

Calcite, with its JSON driver, could effectively enable any application to 
safely expose SQL-based queries, and remove the need for limited-capability 
domain specific languages. 

Calcite can already restrict queries to whitelisted operators, and the 
optimiser can deftly handle a bag of tenancy-related constraints.

It would be possible to extend further by clamping query COST, etc.

E.g. this capability has been productised by stripe, 
https://stripe.com/us/sigma.

Cheers,
Mark.


> On 25 Feb 2018, at 08:24, Michael Mior  wrote:
> 
> You have probably seen that Apache was accepted as an organization for this
> year's GSoC. I thought I would see if anyone in the Calcite community can
> think of any issues that would be a good fit. It's no guarantee we would
> get someone to work on it, but it could be a good push to move some
> isolated bits of functionality forward that may not get much attention
> otherwise.
> 
> --
> Michael Mior
> mm...@apache.org

Re: [DISCUSS] Towards Avatica 1.11.0

[F21]

Also would like to see 
https://issues.apache.org/jira/projects/CALCITE/issues/CALCITE-1951 
worked on.


Francis

On 28/02/2018 9:34 AM, Julian Hyde wrote:

OK, new thread just for the Avatica release. What’s left to do before 1.11? 
What can we accomplish before Monday?

There are 20 open JIRA cases tagged for 1.11: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%20avatica-1.11.0%20and%20status%20!%3D%20Resolved%20
 
.

Josh, 12 of those issues are assigned to you. Do you intend to fix any of those? If 
so, make them dependencies of https://issues.apache.org/jira/browse/CALCITE-2182 
.

Haohui, you have https://issues.apache.org/jira/browse/CALCITE-1884 
. Please update the case 
when you make progress.

I’ll take a shot at https://issues.apache.org/jira/browse/CALCITE-1928 
, review 
https://github.com/apache/calcite-avatica/pull/23 
 = 
https://issues.apache.org/jira/browse/CALCITE-508 
, commit 
https://github.com/apache/calcite-avatica/pull/16 
, and of course write the release notes.

Anything else that needs to be done, or any fixes/pull requests that people 
would like to get in?

Julian

Re: [DISCUSS] Towards Avatica 1.11.0

[F21]

Do you guys think you can work on 
https://issues.apache.org/jira/projects/CALCITE/issues/CALCITE-2013 ?


I tried bumping HSQLDB in the pom.xml a few months ago, but there were a 
few failing tests. Bumping HSQLDB will solve a bunch of issues I saw 
when testing avatica-go against avatica with HSQLDB. This should put us 
closer to a release for avatica-go.


Francis

On 28/02/2018 9:34 AM, Julian Hyde wrote:

OK, new thread just for the Avatica release. What’s left to do before 1.11? 
What can we accomplish before Monday?

There are 20 open JIRA cases tagged for 1.11: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%20avatica-1.11.0%20and%20status%20!%3D%20Resolved%20
 
.

Josh, 12 of those issues are assigned to you. Do you intend to fix any of those? If 
so, make them dependencies of https://issues.apache.org/jira/browse/CALCITE-2182 
.

Haohui, you have https://issues.apache.org/jira/browse/CALCITE-1884 
. Please update the case 
when you make progress.

I’ll take a shot at https://issues.apache.org/jira/browse/CALCITE-1928 
, review 
https://github.com/apache/calcite-avatica/pull/23 
 = 
https://issues.apache.org/jira/browse/CALCITE-508 
, commit 
https://github.com/apache/calcite-avatica/pull/16 
, and of course write the release notes.

Anything else that needs to be done, or any fixes/pull requests that people 
would like to get in?

Julian

[DISCUSS] Towards Avatica 1.11.0

[Julian Hyde]

OK, new thread just for the Avatica release. What’s left to do before 1.11? 
What can we accomplish before Monday?

There are 20 open JIRA cases tagged for 1.11: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%20avatica-1.11.0%20and%20status%20!%3D%20Resolved%20
 
.

Josh, 12 of those issues are assigned to you. Do you intend to fix any of 
those? If so, make them dependencies of 
https://issues.apache.org/jira/browse/CALCITE-2182 
.

Haohui, you have https://issues.apache.org/jira/browse/CALCITE-1884 
. Please update the case 
when you make progress.

I’ll take a shot at https://issues.apache.org/jira/browse/CALCITE-1928 
, review 
https://github.com/apache/calcite-avatica/pull/23 
 = 
https://issues.apache.org/jira/browse/CALCITE-508 
, commit 
https://github.com/apache/calcite-avatica/pull/16 
, and of course write the 
release notes.

Anything else that needs to be done, or any fixes/pull requests that people 
would like to get in?

Julian

Re: [DISCUSS] Towards Calcite 1.16.0

[Jesus Camacho Rodriguez]

Thanks Julian.

I did not reply to previous email, but plan sounded reasonable to me. I will be 
ready to have the first Calcite RC shortly after Avatica is released.

-Jesús


On 2/27/18, 11:49 AM, "Julian Hyde"  wrote:

Did we agree a target date for Calcite 1.16?

In any case, I think we’d better do an Avatica release very soon. I 
volunteer to be release manager for Avatica. Can we aim for an RC of Avatica 
1.11 on Monday (March 5)?

Then, if all goes well, the first RC for Calcite 1.16 could be as early as 
Monday March 12. As RM, I’ll let Jesus drive the Calcite release.

Julian


> On Feb 17, 2018, at 2:44 AM, Chris Baynes  wrote:
> 
> New release would be great!
> I'd very much like to get the ClickHouse dialect PR in the next release:
> https://issues.apache.org/jira/browse/CALCITE-2157
> though I haven't figured out an easy way to integrate this into the 
calcite
> test dataset (commented about this in the jira issue).
> 
> On Sat, Feb 17, 2018 at 9:22 AM, Shuyi Chen  wrote:
> 
>> Thanks a lot, Julian. I think I can try to make it before Mid March.
>> 
>> On Fri, Feb 16, 2018 at 2:46 PM, Julian Hyde  wrote:
>> 
>>> Shuyi,
>>> 
>>> I forgot to ask: What would be your preferred time-frame for the 
release?
>>> 
>>> Julian
>>> 
>>> 
 On Feb 16, 2018, at 10:59 AM, Julian Hyde  wrote:
 
 My preferred time-frame would be to start a vote on Mar 1 or soon
>> after.
>>> So we’d have a release by say Mar 10. That gives us 2 weeks to produce 
an
>>> Avatica release. But I’m flexible - the release could happen a couple of
>>> weeks later in March.
 
 In my queue I have the Geode and Jethro adapters. Both are almost ready
>>> to commit - I just need to get clean test runs. I am interested in
>>> finishing https://issues.apache.org/jira/browse/CALCITE-2160 <
>>> https://issues.apache.org/jira/browse/CALCITE-2160> (a grid index to
>>> support spatial joins) or at least some blocker issues regarding CROSS
>>> APPLY that I ran into while working on this.
 
 Take a look at the pull request queue https://github.com/apache/
>>> calcite/pulls  - there are a
>> lot
>>> of half-complete PRs, more than usual. If the authors want make a
>> concerted
>>> effort to complete them before the release they should speak up now, and
>> we
>>> can be flexible.
 
 Julian
 
> On Feb 16, 2018, at 10:51 AM, Shuyi Chen > suez1...@gmail.com>> wrote:
> 
> I am working on https://issues.apache.org/jira/browse/CALCITE-2045 <
>>> https://issues.apache.org/jira/browse/CALCITE-2045> (Create
> Type DDL). I was hoping to get it in for 1.16. Do we have the deadline
>>> for
> the 1.16 feature cut? Thanks.
> 
> On Fri, Feb 16, 2018 at 10:43 AM, Julian Hyde >> > wrote:
> 
>> Sounds good. It’s about time for 1.16, and thanks for offering to be
>> release manager.
>> 
>> But I’d like to get Avatica 1.11 [1] released first — it’s been 9
>>> months —
>> and make Calcite 1.16 depend on Avatica 1.11. See email thread[2].
>>> Although
>> that thread talks about an Avatica-Go release, that should not be a
>>> blocker
>> for Calcite 1.16. For both Avatica and Calcite we should update
>> dependencies.
>> 
>> Julian
>> 
>> [1] https://issues.apache.org/jira/browse/

CALCITE-2182 <
>>> https://issues.apache.org/jira/browse/CALCITE-2182> <
>> https://issues.apache.org/jira/browse/CALCITE-2182 <
>>> https://issues.apache.org/jira/browse/CALCITE-2182>>
>> 
>> [2] https://lists.apache.org/thread.html/
>>> a72dbd9103489058562f26a943f7f5 >> a72dbd9103489058562f26a943f7f5>
>> 84e10a0cd04e0e5f69b00c0e49@%3Cdev.calcite.apache.org <
>>> http://3cdev.calcite.apache.org/>%3E <
>> https://lists.apache.org/thread.html/a72dbd9103489058562f26a943f7f5
>> <
>>> https://lists.apache.org/thread.html/a72dbd9103489058562f26a943f7f5>
>> 84e10a0cd04e0e5f69b00c0e49@%3Cdev.calcite.apache.org <
>>> http://3cdev.calcite.apache.org/>%3E>
>> 
>> 
>>> On Feb 16, 2018, at 10:25 AM, Jesus Camacho Rodriguez <
>> jcama...@apache.org > wrote:
>>> 
>>> Calcite 1.15.0 was released on December 11 (more than two months
>> ago).
>>> We have solved over 50 issues since then, hence I think we should
>>> start
>>> discussing about releasing Calcite 

Re: [DISCUSS] Towards Calcite 1.16.0

[Julian Hyde]

Did we agree a target date for Calcite 1.16?

In any case, I think we’d better do an Avatica release very soon. I volunteer 
to be release manager for Avatica. Can we aim for an RC of Avatica 1.11 on 
Monday (March 5)?

Then, if all goes well, the first RC for Calcite 1.16 could be as early as 
Monday March 12. As RM, I’ll let Jesus drive the Calcite release.

Julian


> On Feb 17, 2018, at 2:44 AM, Chris Baynes  wrote:
> 
> New release would be great!
> I'd very much like to get the ClickHouse dialect PR in the next release:
> https://issues.apache.org/jira/browse/CALCITE-2157
> though I haven't figured out an easy way to integrate this into the calcite
> test dataset (commented about this in the jira issue).
> 
> On Sat, Feb 17, 2018 at 9:22 AM, Shuyi Chen  wrote:
> 
>> Thanks a lot, Julian. I think I can try to make it before Mid March.
>> 
>> On Fri, Feb 16, 2018 at 2:46 PM, Julian Hyde  wrote:
>> 
>>> Shuyi,
>>> 
>>> I forgot to ask: What would be your preferred time-frame for the release?
>>> 
>>> Julian
>>> 
>>> 
 On Feb 16, 2018, at 10:59 AM, Julian Hyde  wrote:
 
 My preferred time-frame would be to start a vote on Mar 1 or soon
>> after.
>>> So we’d have a release by say Mar 10. That gives us 2 weeks to produce an
>>> Avatica release. But I’m flexible - the release could happen a couple of
>>> weeks later in March.
 
 In my queue I have the Geode and Jethro adapters. Both are almost ready
>>> to commit - I just need to get clean test runs. I am interested in
>>> finishing https://issues.apache.org/jira/browse/CALCITE-2160 <
>>> https://issues.apache.org/jira/browse/CALCITE-2160> (a grid index to
>>> support spatial joins) or at least some blocker issues regarding CROSS
>>> APPLY that I ran into while working on this.
 
 Take a look at the pull request queue https://github.com/apache/
>>> calcite/pulls  - there are a
>> lot
>>> of half-complete PRs, more than usual. If the authors want make a
>> concerted
>>> effort to complete them before the release they should speak up now, and
>> we
>>> can be flexible.
 
 Julian
 
> On Feb 16, 2018, at 10:51 AM, Shuyi Chen > suez1...@gmail.com>> wrote:
> 
> I am working on https://issues.apache.org/jira/browse/CALCITE-2045 <
>>> https://issues.apache.org/jira/browse/CALCITE-2045> (Create
> Type DDL). I was hoping to get it in for 1.16. Do we have the deadline
>>> for
> the 1.16 feature cut? Thanks.
> 
> On Fri, Feb 16, 2018 at 10:43 AM, Julian Hyde >> > wrote:
> 
>> Sounds good. It’s about time for 1.16, and thanks for offering to be
>> release manager.
>> 
>> But I’d like to get Avatica 1.11 [1] released first — it’s been 9
>>> months —
>> and make Calcite 1.16 depend on Avatica 1.11. See email thread[2].
>>> Although
>> that thread talks about an Avatica-Go release, that should not be a
>>> blocker
>> for Calcite 1.16. For both Avatica and Calcite we should update
>> dependencies.
>> 
>> Julian
>> 
>> [1] https://issues.apache.org/jira/browse/CALCITE-2182 <
>>> https://issues.apache.org/jira/browse/CALCITE-2182> <
>> https://issues.apache.org/jira/browse/CALCITE-2182 <
>>> https://issues.apache.org/jira/browse/CALCITE-2182>>
>> 
>> [2] https://lists.apache.org/thread.html/
>>> a72dbd9103489058562f26a943f7f5 >> a72dbd9103489058562f26a943f7f5>
>> 84e10a0cd04e0e5f69b00c0e49@%3Cdev.calcite.apache.org <
>>> http://3cdev.calcite.apache.org/>%3E <
>> https://lists.apache.org/thread.html/a72dbd9103489058562f26a943f7f5
>> <
>>> https://lists.apache.org/thread.html/a72dbd9103489058562f26a943f7f5>
>> 84e10a0cd04e0e5f69b00c0e49@%3Cdev.calcite.apache.org <
>>> http://3cdev.calcite.apache.org/>%3E>
>> 
>> 
>>> On Feb 16, 2018, at 10:25 AM, Jesus Camacho Rodriguez <
>> jcama...@apache.org > wrote:
>>> 
>>> Calcite 1.15.0 was released on December 11 (more than two months
>> ago).
>>> We have solved over 50 issues since then, hence I think we should
>>> start
>>> discussing about releasing Calcite 1.16.0 (I can be release manager
>> if
>>> nobody else steps up).
>>> 
>>> I have created [1] to keep track.
>>> 
>>> CALCITE-2027 (dropping JDK7 support) was targeted for 1.16.0.
>> Julian,
>>> Michael, you have been working on that issue, could you share the
>>> current status?
>>> Any other particular features that people would like to include?
>>> 
>>> -Jesús
>>> 
>>> [1] https://issues.apache.org/jira/browse/CALCITE-2181 <
>>> https://issues.apache.org/jira/browse/CALCITE-2181>
>>> 
>>> 
>> 
>> 
> 
> 
> --
> "So you have to trust that the dots will somehow connect in your
>>> 

Re: Google Summer of Code 2018

[Michael Mior]

While it would be great to have some movement forward on this, I think it
seems too ambitious for a GSoC project. I am definitely looking forward to
seeing this implemented though :)

As for the issues I referenced earlier, I applied the GSoC label and below
is a short link. Feel free to pass around to anyone you know who may be
interested.

https://s.apache.org/calcite-gsoc2018

Cheers,
--
Michael Mior
mm...@apache.org

2018-02-27 10:24 GMT-05:00 Edmon Begoli :

> If we are to be ambitious, and to give someone a real challenge --
> implement this approach for Calcite:
> Multi-objective Parametric Query Optimization
> https://cacm.acm.org/magazines/2017/10/221322-multi-objective-parametric-
> query-optimization/fulltext
>
> (I think Julian might have mentioned this once before in a different
> context. I might be wrong ...)
>
> On Tue, Feb 27, 2018 at 10:20 AM, Michael Mior  wrote:
>
> > I would suggest creating a JIRA case to track this. It sounds like a
> basic
> > implementation could be of reasonable scope for a GSoC project.
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > 2018-02-26 19:09 GMT-05:00 Chunhui Shi :
> >
> > > What about "SQL support for JavaScript Object Notation (JSON)"?
> > > www.iso.org/standard/67367.html >?
> > >
> > > There are two categories of JSON functions (8 functions) for query and
> > for
> > > constructor, some clauses like PLAN, etc are introduced in this
> standard.
> > >
> > > I did not see the supports in mainstream DBs(Oracle and Sql Server)
> > > covering all of them, but IS JSON, JSON_QUERY and JSON_VALUE are both
> > > implemented
> > >
> > > SQL server support
> > > https://docs.microsoft.com/en-us/sql/relational-databases/
> > > json/validate-query-and-change-json-data-with-built-
> > > in-functions-sql-server
> > >
> > > Oracle:
> > > https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB6374
> > >
> > > MySQL and Postgres have a lot functions to handle JSON,  besides
> > > JSON_OBJECT or one or two other functions, seems none of its function
> is
> > > standardized.
> > > https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html
> > >
> > > Postgres
> > > 
> > > https://www.postgresql.org/docs/current/static/functions-json.html
> > >
> > >
> > > While I think the scope could be self contained, if it is not suitable
> > for
> > > this activity do we want to put this into Calcite roadmap?
> > >
> > >
> > > Best,
> > >
> > > Chunhui
> > >
> > > 
> > > From: Julian Hyde 
> > > Sent: Monday, February 26, 2018 4:00:21 PM
> > > To: dev
> > > Subject: Re: Google Summer of Code 2018
> > >
> > > Yes.
> > >
> > >
> > > > On Feb 26, 2018, at 2:32 PM, Michael Mior 
> wrote:
> > > >
> > > > Thanks Julian! Would these be some good JIRAs to tag?
> > > >
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > apache.org_jira_browse_CALCITE-2D1737=DwICAg=
> > cskdkSMqhcnjZxdQVpwTXg=
> > > 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-
> eWRg5qGj9K6txN2zGGcqakg=
> > > KXsK-qlYTOf5W1KDQmlqjA7CIj4_CtGlmWirBlPxopI=
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > apache.org_jira_browse_CALCITE-2D1861=DwICAg=
> > cskdkSMqhcnjZxdQVpwTXg=
> > > 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-
> eWRg5qGj9K6txN2zGGcqakg=
> > > 6xfRRfhSus-SrVHKvEIYKzcyAaEAUJBLdHbkqjUhrRk=
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > > apache.org_jira_browse_CALCITE-2D2031=DwICAg=
> > cskdkSMqhcnjZxdQVpwTXg=
> > > 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-
> > > eWRg5qGj9K6txN2zGGcqakg=oLu_OiKf-cFe1RxcmAnv2CFW6QPGbpz5uYeiN5y
> 3EMs=
> > > >
> > > > --
> > > > Michael Mior
> > > > mm...@apache.org
> > > >
> > > > 2018-02-26 14:46 GMT-05:00 Julian Hyde :
> > > >
> > > >> Here are two areas that are self-contained and rewarding:
> > > >> * Spatial functions
> > > >> * Spark adapter
> > > >>
> > > >> Julian
> > > >>
> > > >>
> > > >>
> > > >>> On Feb 24, 2018, at 4:24 PM, Michael Mior 
> > wrote:
> > > >>>
> > > >>> You have probably seen that Apache was accepted as an organization
> > for
> > > >> this
> > > >>> year's GSoC. I thought I would see if anyone in the Calcite
> community
> > > can
> > > >>> think of any issues that would be a good fit. It's no guarantee we
> > > would
> > > >>> get someone to work on it, but it could be a good push to move some
> > > >>> isolated bits of functionality forward that may not get much
> > attention
> > > >>> otherwise.
> > > >>>
> > > >>> --
> > > >>> Michael Mior
> > > >>> mm...@apache.org
> > > >>
> > > >>
> > >
> > >
> >
>

Re: Google Summer of Code 2018

[Edmon Begoli]

If we are to be ambitious, and to give someone a real challenge --
implement this approach for Calcite:
Multi-objective Parametric Query Optimization
https://cacm.acm.org/magazines/2017/10/221322-multi-objective-parametric-query-optimization/fulltext

(I think Julian might have mentioned this once before in a different
context. I might be wrong ...)

On Tue, Feb 27, 2018 at 10:20 AM, Michael Mior  wrote:

> I would suggest creating a JIRA case to track this. It sounds like a basic
> implementation could be of reasonable scope for a GSoC project.
>
> --
> Michael Mior
> mm...@apache.org
>
> 2018-02-26 19:09 GMT-05:00 Chunhui Shi :
>
> > What about "SQL support for JavaScript Object Notation (JSON)"?
> > www.iso.org/standard/67367.html?
> >
> > There are two categories of JSON functions (8 functions) for query and
> for
> > constructor, some clauses like PLAN, etc are introduced in this standard.
> >
> > I did not see the supports in mainstream DBs(Oracle and Sql Server)
> > covering all of them, but IS JSON, JSON_QUERY and JSON_VALUE are both
> > implemented
> >
> > SQL server support
> > https://docs.microsoft.com/en-us/sql/relational-databases/
> > json/validate-query-and-change-json-data-with-built-
> > in-functions-sql-server
> >
> > Oracle:
> > https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB6374
> >
> > MySQL and Postgres have a lot functions to handle JSON,  besides
> > JSON_OBJECT or one or two other functions, seems none of its function is
> > standardized.
> > https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html
> >
> > Postgres
> > 
> > https://www.postgresql.org/docs/current/static/functions-json.html
> >
> >
> > While I think the scope could be self contained, if it is not suitable
> for
> > this activity do we want to put this into Calcite roadmap?
> >
> >
> > Best,
> >
> > Chunhui
> >
> > 
> > From: Julian Hyde 
> > Sent: Monday, February 26, 2018 4:00:21 PM
> > To: dev
> > Subject: Re: Google Summer of Code 2018
> >
> > Yes.
> >
> >
> > > On Feb 26, 2018, at 2:32 PM, Michael Mior  wrote:
> > >
> > > Thanks Julian! Would these be some good JIRAs to tag?
> > >
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CALCITE-2D1737=DwICAg=
> cskdkSMqhcnjZxdQVpwTXg=
> > 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-eWRg5qGj9K6txN2zGGcqakg=
> > KXsK-qlYTOf5W1KDQmlqjA7CIj4_CtGlmWirBlPxopI=
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CALCITE-2D1861=DwICAg=
> cskdkSMqhcnjZxdQVpwTXg=
> > 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-eWRg5qGj9K6txN2zGGcqakg=
> > 6xfRRfhSus-SrVHKvEIYKzcyAaEAUJBLdHbkqjUhrRk=
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CALCITE-2D2031=DwICAg=
> cskdkSMqhcnjZxdQVpwTXg=
> > 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-
> > eWRg5qGj9K6txN2zGGcqakg=oLu_OiKf-cFe1RxcmAnv2CFW6QPGbpz5uYeiN5y3EMs=
> > >
> > > --
> > > Michael Mior
> > > mm...@apache.org
> > >
> > > 2018-02-26 14:46 GMT-05:00 Julian Hyde :
> > >
> > >> Here are two areas that are self-contained and rewarding:
> > >> * Spatial functions
> > >> * Spark adapter
> > >>
> > >> Julian
> > >>
> > >>
> > >>
> > >>> On Feb 24, 2018, at 4:24 PM, Michael Mior 
> wrote:
> > >>>
> > >>> You have probably seen that Apache was accepted as an organization
> for
> > >> this
> > >>> year's GSoC. I thought I would see if anyone in the Calcite community
> > can
> > >>> think of any issues that would be a good fit. It's no guarantee we
> > would
> > >>> get someone to work on it, but it could be a good push to move some
> > >>> isolated bits of functionality forward that may not get much
> attention
> > >>> otherwise.
> > >>>
> > >>> --
> > >>> Michael Mior
> > >>> mm...@apache.org
> > >>
> > >>
> >
> >
>

Re: Google Summer of Code 2018

[Michael Mior]

I would suggest creating a JIRA case to track this. It sounds like a basic
implementation could be of reasonable scope for a GSoC project.

--
Michael Mior
mm...@apache.org

2018-02-26 19:09 GMT-05:00 Chunhui Shi :

> What about "SQL support for JavaScript Object Notation (JSON)"?
> www.iso.org/standard/67367.html?
>
> There are two categories of JSON functions (8 functions) for query and for
> constructor, some clauses like PLAN, etc are introduced in this standard.
>
> I did not see the supports in mainstream DBs(Oracle and Sql Server)
> covering all of them, but IS JSON, JSON_QUERY and JSON_VALUE are both
> implemented
>
> SQL server support
> https://docs.microsoft.com/en-us/sql/relational-databases/
> json/validate-query-and-change-json-data-with-built-
> in-functions-sql-server
>
> Oracle:
> https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB6374
>
> MySQL and Postgres have a lot functions to handle JSON,  besides
> JSON_OBJECT or one or two other functions, seems none of its function is
> standardized.
> https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html
>
> Postgres
> 
> https://www.postgresql.org/docs/current/static/functions-json.html
>
>
> While I think the scope could be self contained, if it is not suitable for
> this activity do we want to put this into Calcite roadmap?
>
>
> Best,
>
> Chunhui
>
> 
> From: Julian Hyde 
> Sent: Monday, February 26, 2018 4:00:21 PM
> To: dev
> Subject: Re: Google Summer of Code 2018
>
> Yes.
>
>
> > On Feb 26, 2018, at 2:32 PM, Michael Mior  wrote:
> >
> > Thanks Julian! Would these be some good JIRAs to tag?
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_CALCITE-2D1737=DwICAg=cskdkSMqhcnjZxdQVpwTXg=
> 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-eWRg5qGj9K6txN2zGGcqakg=
> KXsK-qlYTOf5W1KDQmlqjA7CIj4_CtGlmWirBlPxopI=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_CALCITE-2D1861=DwICAg=cskdkSMqhcnjZxdQVpwTXg=
> 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-eWRg5qGj9K6txN2zGGcqakg=
> 6xfRRfhSus-SrVHKvEIYKzcyAaEAUJBLdHbkqjUhrRk=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_CALCITE-2D2031=DwICAg=cskdkSMqhcnjZxdQVpwTXg=
> 7bZjGOKpQi7qeyQ_xRpzEQ=LYUbb9SvNkKb5PpYzt7-
> eWRg5qGj9K6txN2zGGcqakg=oLu_OiKf-cFe1RxcmAnv2CFW6QPGbpz5uYeiN5y3EMs=
> >
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > 2018-02-26 14:46 GMT-05:00 Julian Hyde :
> >
> >> Here are two areas that are self-contained and rewarding:
> >> * Spatial functions
> >> * Spark adapter
> >>
> >> Julian
> >>
> >>
> >>
> >>> On Feb 24, 2018, at 4:24 PM, Michael Mior  wrote:
> >>>
> >>> You have probably seen that Apache was accepted as an organization for
> >> this
> >>> year's GSoC. I thought I would see if anyone in the Calcite community
> can
> >>> think of any issues that would be a good fit. It's no guarantee we
> would
> >>> get someone to work on it, but it could be a good push to move some
> >>> isolated bits of functionality forward that may not get much attention
> >>> otherwise.
> >>>
> >>> --
> >>> Michael Mior
> >>> mm...@apache.org
> >>
> >>
>
>

[jira] [Created] (CALCITE-2195) AggregateJoinTransposeRule fails to aggregate over unique column

[Zhong Yu (JIRA)]

Zhong Yu created CALCITE-2195:
-

 Summary: AggregateJoinTransposeRule fails to aggregate over unique 
column
 Key: CALCITE-2195
 URL: https://issues.apache.org/jira/browse/CALCITE-2195
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.15.0
Reporter: Zhong Yu
Assignee: Julian Hyde
 Fix For: 1.16.0


The following query, in which "A.sal" is unique,
{code:java}
select sum(A.sal)
from (select distinct sal from sales.emp) as A
join sales.emp as B on A.sal=B.sal
{code}
 causes AggregateJoinTransposeRule to fail with message
{code:java}
java.lang.AssertionError: type mismatch:
aggCall type:
INTEGER
inferred type:
BIGINT
{code}
Apparently, this is a bug in the rule when `unique` is true on the A side, in 
which case the rule does not aggregate on the A side, `leftSubTotal==null`, 
causing `splitter.topSplit()` to only sum over `count(*)` coming from the B 
side.

A solution would be to introduce `splitter.singleton()` on the A side, so that 
it can be fed to topSplit() to be multiplied by the count.

In the case that the `unique` side does not contain the column of an agg call, 
it seems that we should do `other_singleton()` on this side, and feed it to 
topSplit(). However, realize that the `other()` expression is necessarily a 
`count(*)`, or a scalar function of `count(*)`, because it does not depend on 
any column values. In the same way, the proposed `other_singleton()` 
necessarily returns 1, or some constant value. `topSplit()` would not have any 
need of that constant value.Therefore in this case, we don't need a split on 
this side, just leave its subtotal as null.

 

I'm working on a pull-request based on these understandings. BTW, is there a 
reference to the algorithm used in the code? I can only find some old papers 
that don't exactly cover the logic of the code. Thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)