Re: Blocker on drill upgrade path

2019-04-22 Thread Vova Vysotskyi 
Hi Nitin,

This behavior to allow aliases in a group by clause is driven by Calcite
and commonly used in other projects.
I think the workaround proposed by Aman is the best solution for this
problem, since for example if you have several aggregate functions in the
project for the same columns, it will cause problems with such naming.

Kind regards,
Volodymyr Vysotskyi


On Sat, Apr 20, 2019 at 8:44 AM Nitin Pawar  wrote:

> Right now the aliases are derived programmatically and we use the same name
> in group by as an alias and these are already defined in the jobs so we can
> not change them now
> That's one reason it became blocker as these jobs are configured and were
> running fine and suddenly started breaking.
>
> On Sat, Apr 20, 2019 at 5:24 AM Aman Sinha  wrote:
>
> > Interesting that it ran on 1.13..but I still think the new behavior is
> the
> > right one.  Several changes went into Calcite between Drill's 1.13 and
> 1.15
> > release, so I cannot identify when this behavior changed.   Can you use a
> > slightly different alias name ?  The following should work:
> > select max(last_name) *max_last_name* from cp.`employee.json` group
> by
> > last_name limit 5;
> >
> > On Fri, Apr 19, 2019 at 2:24 PM Nitin Pawar 
> > wrote:
> >
> > > sorry  my bad. i meant the query which was failing was with alias
> > > following is output on drill 1.13.0
> > >
> > > bash-3.2$ bin/drill-embedded
> > > Apr 20, 2019 2:46:45 AM org.glassfish.jersey.server.ApplicationHandler
> > > initialize
> > > INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> > > 01:25:26...
> > > apache drill 1.13.0-SNAPSHOT
> > > "a drill in the hand is better than two in the bush"
> > > 0: jdbc:drill:zk=local> select max(last_name) last_name from
> > > cp.`employee.json` group by
> > > . . . . . . . . . . . > last_name limit 5;
> > > ++
> > > | last_name  |
> > > ++
> > > | Nowmer |
> > > | Whelply|
> > > | Spence |
> > > | Gutierrez  |
> > > | Damstra|
> > > ++
> > >
> > >
> > > On Sat, Apr 20, 2019 at 1:40 AM Aman Sinha 
> wrote:
> > >
> > > > This is legal:
> > > >   select max(last_name)  from cp.`employee.json` group by last_name
> > limit
> > > > 5;
> > > > But this is not:
> > > >   select max(last_name) last_name from cp.`employee.json` group by
> > > > last_name limit 5;
> > > >
> > > > The reason is the second query is aliasing the max() output to
> > > 'last_name'
> > > > which is being referenced in the group-by clause.  Referencing an
> > > aggregate
> > > > expr in the group-by is not allowed by SQL standards, hence Calcite
> > > (which
> > > > does the parsing and validation, not Drill) throws this error during
> > > > validation phase.  Detailed error stack is below.  I don't think this
> > > would
> > > > have worked in 1.13 either.  My guess is you may have run the first
> > query
> > > > in 1.13 and that should still continue to work.
> > > >
> > > > Validation error thrown by Calcite:
> > > >
> > > > Caused By (org.apache.calcite.sql.validate.SqlValidatorException)
> > > Aggregate
> > > > expression is illegal in GROUP BY clause
> > > >
> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
> > > >
> > > > sun.reflect.NativeConstructorAccessorImpl.newInstance():62
> > > >
> > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
> > > >
> > > > java.lang.reflect.Constructor.newInstance():423
> > > >
> > > > org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
> > > >
> > > > org.apache.calcite.runtime.Resources$ExInst.ex():572
> > > >
> > > > org.apache.calcite.sql.SqlUtil.newContextException():787
> > > >
> > > > org.apache.calcite.sql.SqlUtil.newContextException():772
> > > >
> > > >
> > > >
> > >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4788
> > > >
> > > >
> > > >
> > >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupClause():3941
> > > >
> > > >
> > >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3306
> > > >
> > > > org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
> > > >
> > > > org.apache.calcite.sql.validate.AbstractNamespace.validate():84
> > > >
> > > >
> > > >
> > org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():977
> > > >
> > > >
> >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():953
> > > >
> > > > org.apache.calcite.sql.SqlSelect.validate():216
> > > >
> > > >
> > > >
> > > >
> > >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():928
> > > >
> > > > org.apache.calcite.sql.validate.SqlValidatorImpl.validate():632
> > > >
> > > > org.apache.drill.exec.planner.sql.SqlConverter.validate():207
> > > >
> > > > On Fri, Apr 19, 2019 at 12:39 PM Nitin Pawar <
> nitinpawar...@gmail.com>
> > > > wrote:
> > > >
> > > > > I think the error is not with storage plugin but with query parsing
> > > > >

Re: Blocker on drill upgrade path

2019-04-22 Thread Nitin Pawar 
Agreed.
I will see what kind of alternative we can do on our side

On Mon, Apr 22, 2019 at 2:53 PM Vova Vysotskyi  wrote:

> Hi Nitin,
>
> This behavior to allow aliases in a group by clause is driven by Calcite
> and commonly used in other projects.
> I think the workaround proposed by Aman is the best solution for this
> problem, since for example if you have several aggregate functions in the
> project for the same columns, it will cause problems with such naming.
>
> Kind regards,
> Volodymyr Vysotskyi
>
>
> On Sat, Apr 20, 2019 at 8:44 AM Nitin Pawar 
> wrote:
>
> > Right now the aliases are derived programmatically and we use the same
> name
> > in group by as an alias and these are already defined in the jobs so we
> can
> > not change them now
> > That's one reason it became blocker as these jobs are configured and were
> > running fine and suddenly started breaking.
> >
> > On Sat, Apr 20, 2019 at 5:24 AM Aman Sinha  wrote:
> >
> > > Interesting that it ran on 1.13..but I still think the new behavior is
> > the
> > > right one.  Several changes went into Calcite between Drill's 1.13 and
> > 1.15
> > > release, so I cannot identify when this behavior changed.   Can you
> use a
> > > slightly different alias name ?  The following should work:
> > > select max(last_name) *max_last_name* from cp.`employee.json` group
> > by
> > > last_name limit 5;
> > >
> > > On Fri, Apr 19, 2019 at 2:24 PM Nitin Pawar 
> > > wrote:
> > >
> > > > sorry  my bad. i meant the query which was failing was with alias
> > > > following is output on drill 1.13.0
> > > >
> > > > bash-3.2$ bin/drill-embedded
> > > > Apr 20, 2019 2:46:45 AM
> org.glassfish.jersey.server.ApplicationHandler
> > > > initialize
> > > > INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29
> > > > 01:25:26...
> > > > apache drill 1.13.0-SNAPSHOT
> > > > "a drill in the hand is better than two in the bush"
> > > > 0: jdbc:drill:zk=local> select max(last_name) last_name from
> > > > cp.`employee.json` group by
> > > > . . . . . . . . . . . > last_name limit 5;
> > > > ++
> > > > | last_name  |
> > > > ++
> > > > | Nowmer |
> > > > | Whelply|
> > > > | Spence |
> > > > | Gutierrez  |
> > > > | Damstra|
> > > > ++
> > > >
> > > >
> > > > On Sat, Apr 20, 2019 at 1:40 AM Aman Sinha 
> > wrote:
> > > >
> > > > > This is legal:
> > > > >   select max(last_name)  from cp.`employee.json` group by last_name
> > > limit
> > > > > 5;
> > > > > But this is not:
> > > > >   select max(last_name) last_name from cp.`employee.json` group by
> > > > > last_name limit 5;
> > > > >
> > > > > The reason is the second query is aliasing the max() output to
> > > > 'last_name'
> > > > > which is being referenced in the group-by clause.  Referencing an
> > > > aggregate
> > > > > expr in the group-by is not allowed by SQL standards, hence Calcite
> > > > (which
> > > > > does the parsing and validation, not Drill) throws this error
> during
> > > > > validation phase.  Detailed error stack is below.  I don't think
> this
> > > > would
> > > > > have worked in 1.13 either.  My guess is you may have run the first
> > > query
> > > > > in 1.13 and that should still continue to work.
> > > > >
> > > > > Validation error thrown by Calcite:
> > > > >
> > > > > Caused By (org.apache.calcite.sql.validate.SqlValidatorException)
> > > > Aggregate
> > > > > expression is illegal in GROUP BY clause
> > > > >
> > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
> > > > >
> > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance():62
> > > > >
> > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
> > > > >
> > > > > java.lang.reflect.Constructor.newInstance():423
> > > > >
> > > > > org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463
> > > > >
> > > > > org.apache.calcite.runtime.Resources$ExInst.ex():572
> > > > >
> > > > > org.apache.calcite.sql.SqlUtil.newContextException():787
> > > > >
> > > > > org.apache.calcite.sql.SqlUtil.newContextException():772
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4788
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupClause():3941
> > > > >
> > > > >
> > > >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3306
> > > > >
> > > > >
>  org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60
> > > > >
> > > > > org.apache.calcite.sql.validate.AbstractNamespace.validate():84
> > > > >
> > > > >
> > > > >
> > >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():977
> > > > >
> > > > >
> > >  org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():953
> > > > >
> > > > > org.apache.calcite.sql.SqlSelect.validate():216
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

[VOTE] Apache Drill Release 1.16.0 - RC1

2019-04-22 Thread SorabhApache 
Hi Drillers,
I'd like to propose the second release candidate (RC1) for the Apache Drill,
version 1.16.0.

Changes since the previous release candidate:
DRILL-7185: Drill Fails to Read Large Packets
DRILL-7186: Missing storage.json REST endpoint
DRILL-7190: Missing backward compatibility for REST API with DRILL-6562

Also below 2 JIRA's were created to separately track revert of protbuf
changes in 1.16.0:
DRILL-7188: Revert DRILL-6642: Update protocol-buffers version
DRILL-7189: Revert DRILL-7105 Error while building the Drill native client

The RC1 includes total of 215 resolved JIRAs [1].
Thanks to everyone for their hard work to contribute to this release.

The tarball artifacts are hosted at [2] and the maven artifacts are hosted
at [3].

This release candidate is based on commit
cf5b758e0a4c22b75bfb02ac2653ff09415ddf53 located at [4].

Please download and try out the release candidate.

The vote ends at 06:00 PM UTC (11:00 AM PDT, 09:00 PM EET, 11:30 PM IST),
Apr 25th, 2019

[ ] +1
[ ] +0
[ ] -1

Here is my vote: +1
  [1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820=12344284
  [2] http://home.apache.org/~sorabh/drill/releases/1.16.0/rc1/

  [3]
https://repository.apache.org/content/repositories/orgapachedrill-1067/
  [4] https://github.com/sohami/drill/commits/drill-1.16.0

Thanks,
Sorabh

Re: [VOTE] Apache Drill Release 1.16.0 - RC1

2019-04-22 Thread SorabhApache 
*< Please disregard previous email, one of the link is not correct in it.
Use the information in this email instead >*

Hi Drillers,
I'd like to propose the second release candidate (RC1) for the Apache Drill,
version 1.16.0.

Changes since the previous release candidate:
DRILL-7185: Drill Fails to Read Large Packets
DRILL-7186: Missing storage.json REST endpoint
DRILL-7190: Missing backward compatibility for REST API with DRILL-6562

Also below 2 JIRA's were created to separately track revert of protbuf
changes in 1.16.0:
DRILL-7188: Revert DRILL-6642: Update protocol-buffers version
DRILL-7189: Revert DRILL-7105 Error while building the Drill native client

The RC1 includes total of 215 resolved JIRAs [1].
Thanks to everyone for their hard work to contribute to this release.

The tarball artifacts are hosted at [2] and the maven artifacts are hosted
at [3].

This release candidate is based on commit
cf5b758e0a4c22b75bfb02ac2653ff09415ddf53 located at [4].

Please download and try out the release candidate.

The vote ends at 06:00 PM UTC (11:00 AM PDT, 09:00 PM EET, 11:30 PM IST),
Apr 25th, 2019

[ ] +1
[ ] +0
[ ] -1

Here is my vote: +1
  [1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820=12344284
  [2] http://home.apache.org/~sorabh/drill/releases/1.16.0/rc1/
  [3]
https://repository.apache.org/content/repositories/orgapachedrill-1067/
  [4] https://github.com/sohami/drill/commits/drill-1.16.0

Thanks,
Sorabh

>