[jira] [Created] (DRILL-6256) Remove references to java 7 from readme and other files

[Volodymyr Tkach (JIRA)]

Volodymyr Tkach created DRILL-6256:
--

 Summary: Remove references to java 7 from readme and other files
 Key: DRILL-6256
 URL: https://issues.apache.org/jira/browse/DRILL-6256
 Project: Apache Drill
  Issue Type: Bug
Reporter: Volodymyr Tkach
Assignee: Volodymyr Tkach


Since master branch uses jdk 8 we should remove all references to java 7.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] drill issue #1158: DRILL-6145: Implement Hive MapR-DB JSON handler

[priteshm]

Github user priteshm commented on the issue:

https://github.com/apache/drill/pull/1158
  
@vdiravka please do link the JIRA that you will create.
@vrozov can you take a final look?


---

[GitHub] drill issue #1171: DRILL-6231: Fix memory allocation for repeated list vecto...

[ppadma]

Github user ppadma commented on the issue:

https://github.com/apache/drill/pull/1171
  
@paul-rogers please review.


---

[GitHub] drill issue #1161: DRILL-6230: Extend row set readers to handle hyper vector...

[ppadma]

Github user ppadma commented on the issue:

https://github.com/apache/drill/pull/1161
  
@paul-rogers I started the review. will get back soon.


---

[jira] [Resolved] (DRILL-520) ceiling/ceil and floor functions return decimal value instead of an integer

[Karthikeyan Manivannan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthikeyan Manivannan resolved DRILL-520.
--
Resolution: Later

> ceiling/ceil and floor functions return decimal value instead of an integer
> ---
>
> Key: DRILL-520
> URL: https://issues.apache.org/jira/browse/DRILL-520
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.0.0
>Reporter: Krystal
>Assignee: Karthikeyan Manivannan
>Priority: Critical
> Fix For: Future
>
> Attachments: DRILL-520.patch
>
>
> Ran the following queries in drill:
> 0: jdbc:drill:schema=dfs> select ceiling(55.8) from dfs.`student` where 
> rownum=11;
> ++
> |   EXPR$0   |
> ++
> | 56.0   |
> ++
> 0: jdbc:drill:schema=dfs> select floor(55.8) from dfs.`student` where 
> rownum=11;
> ++
> |   EXPR$0   |
> ++
> | 55.0   |
> ++
> The same queries executed from oracle, postgres and mysql returned integer 
> values of 56 and 55.
> Found the following description of the two functions from 
> http://users.atw.hu/sqlnut/sqlnut2-chp-4-sect-4.html :
> Ceil/Ceiling:
> Rounds a noninteger value upwards to the next greatest integer. Returns an 
> integer value unchanged.
> Floor:
> Rounds a noninteger value downwards to the next least integer. Returns an 
> integer value unchanged.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] drill pull request #1171: DRILL-6231: Fix memory allocation for repeated lis...

[ppadma]

GitHub user ppadma opened a pull request:

https://github.com/apache/drill/pull/1171

DRILL-6231: Fix memory allocation for repeated list vector



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ppadma/drill DRILL-6231

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1171.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1171


commit 114a526240a272bd626862a02d1acd06b9c8b4cf
Author: Padma Penumarthy 
Date:   2018-03-16T04:50:54Z

DRILL-6231: Fix memory allocation for repeated list vector




---

[GitHub] drill issue #1166: DRILL-6016 - Fix for Error reading INT96 created by Apach...

[rajrahul]

Github user rajrahul commented on the issue:

https://github.com/apache/drill/pull/1166
  
The schema given below creates the issue, as @vdiravka pointed int96 is 
marked required here. This parquet was generated with an older version of spark 
and is included in the test case.

```
message spark_schema {
  optional binary article_no (UTF8);
  optional binary qty (UTF8);
  required int96 run_date;
}

```
Newer spark version created the schema below where int96 has become 
optional.

```
message spark_schema {
  optional binary country (UTF8);
  optional double sales;
  optional int96 targetDate;
}
```


---

Re: [VOTE] Apache Drill release 1.13.0 - RC0

[Vlad Rozov]

0 (non-binding)

- verified release signature and hash
- verified build "mvn -Dmaven.repo.local=/tmp/.m2/repository/ install 
-DskipTests"

- verified NOTICE, LICENSE and README.md
- checked DEPENDENCIES

issues noted (in order of severity)
- the source distribution contains binary jar files under 
exec/java-exec/src/test/resources/jars

- dependency licensed under GNU LGPL
- copyright year range in NOTICE needs to be updated
- release is signed with the signature that is valid till 2018-10-09

Thank you,

Vlad

On 3/15/18 15:31, Sorabh Hamirwasia wrote:

   *   Downloaded binary tarball from [2] and deployed 2 node cluster
   *   Ran some basic queries using sqlline and Web UI with and without 
security enabled
   *   Verified user to bit secure connections using Plain/Kerberos
   *   Verified bit to bit secure connections using Kerberos.
  *   During testing I found an issue that local control message is still 
creating a connection, but is not related to this release. The issue is a regression 
because of state introduced in Drillbit endpoint as part of shutdown feature in 1.12. 
I have opened DRILL-6255 for 
this issue with details in it.
   *   Verified SPNEGO and FORM authentication for Web UI
   *   Ran and verify queries against sys.connections table
   *   Built C++ client on linux box using the source tarball from [2] and ran 
queries to secure and unsecure Drillbit


LGTM +1 (non-binding)


Thanks,
Sorabh


From: Vitalii Diravka 
Sent: Thursday, March 15, 2018 11:12 AM
To: dev@drill.apache.org
Subject: Re: [VOTE] Apache Drill release 1.13.0 - RC0

* Downloaded sources tarball from [2].
Ran drill in embedded mode on local debian machine. Ran tpch queries with
joins, group by, order by, limit, order by with limit statements.
Looked through logs - looks good.
* Build drill for [4] with MapR profile. Ran driillbit in distributed mode
on centos VM with MapR core. Ran queries for Hive 1.2, ran queries for Hive
2.1 (transactional and non-transactional tables). Connected to this
drillbit from remote machine via JDBC with java programm and with SQuirrel
using different drivers (prebuild "drill-jdbc-all-1.12.0.jar" from [2]
tarball, the driver after build [4] sources with default and MapR profiles
too) and ran a simple query. Ran the same with enabled custom
authentication (jdbc driver which is build with MapR profile works good
too).
In the process of testing jdbc connection I found the issue - DRILL-6251.
It is a regression. I have described the case in Jira. But I suppose isn't
critical for current Drill release.
* All unit test were passed for [4]. Total time on my machine was: 42:10 min

+1 (non-binding)

Kind regards
Vitalii

On Thu, Mar 15, 2018 at 4:49 PM, Vova Vysotskyi  wrote:


- Downloaded source tar at [2], ran unit tests and all tests are passed.
- Downloaded built tar at [2], submitted several TPCH queries from UI,
checked that profiles are displayed correctly.
- Connected from SQuirrel, ran several queries; ran queries from a java
application, no issues were found.

+1 (non-binding)


2018-03-15 13:19 GMT+02:00 Arina Yelchiyeva :


  - Built from the source [4] on Linux, run unit test.
- Downloaded the binary tarball [2], untarred and ran Drill in embedded
mode on Windows.
- Ran sample queries, checked system tables, profiles on Web UI, also

logs

and index page.
- Created persistent and temporary tables, loaded custom UDFs.

+1 (binding)

Kind regards
Arina

On Thu, Mar 15, 2018 at 1:39 AM, Aman Sinha 

wrote:

- Downloaded the source tarball from [2] on my Linux VM, built and ran

the

unit tests successfully
- Downloaded the binary tarball onto my Macbook, untarred and ran Drill

in

embedded mode
- Ran several queries  against a TPC-DS SF1 data set, including CTAS
statements with PARTITION BY and ran a few partition pruning queries
- Tested query cancellation by cancelling a query that was taking long

time

due to expanding join
- Examined the run-time query profiles of these queries with and

without

parallelism.
- Checked the maven artifacts on [3].

  - Found one reference to JDK 7 : README.md says 'JDK 7' in the
Prerequisites.  Ideally, this should be changed to JDK 8

Overall, LGTM  +1 (binding)


On Tue, Mar 13, 2018 at 3:58 AM, Parth Chandra 

wrote:

Hi all,

I'd like to propose the first release candidate (RC0) of Apache

Drill,

version 1.13.0.

The release candidate covers a total of 113 resolved JIRAs [1].

Thanks

to everyone
who contributed to this release.

The tarball artifacts are hosted at [2] and the maven artifacts are

hosted

at
[3].

This release candidate is based on commit
cac2882d5a9e22fbc251e4caf622fe30242ad557 located at [4].

Please download and try out the release.

The vote ends at 1:00 PM UTC (5:00 AM PDT, 2:00 PM EET, 5:30 PM IST),

Mar

16th, 2018

[ 

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Paul Rogers]

Congratulations! Thanks for the great contributions.
- Paul

 

On Thursday, March 15, 2018, 10:26:26 AM PDT, Khurram Faraaz 
 wrote:  
 
 Congratulations Volodymyr!


From: Arina Ielchiieva 
Sent: Thursday, March 15, 2018 10:16:51 AM
To: dev@drill.apache.org
Subject: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

The Project Management Committee (PMC) for Apache Drill has
invited Volodymyr Vysotskyi to become a committer, and we are pleased to
announce that he has accepted.

Volodymyr has been contributing for Drill over a year. He contributed in
different areas, including code generation, json processing, function
implementations.
Also he actively participated in Calcite rebase and showed profound knowledge
in planning area.
Currently he is working on decimal's enhancement in Drill.

Congratulations Volodymyr and thank you for your contributions!

- Arina
(on behalf of the Apache Drill PMC)  

[GitHub] drill issue #258: DRILL-4091: Support for additional gis operations in gis c...

[brendanstennett]

Github user brendanstennett commented on the issue:

https://github.com/apache/drill/pull/258
  
Hi @cgivre .  I have some spare cycles coming up in our next open sprint 
starting Apr 2 that either I or someone on my team can take a look at it to get 
it across if it isn't already.  Will keep you updated.  What is your timeline 
for the next release?


---

[GitHub] drill issue #1170: DRILL-6223: Fixed several Drillbit failures due to schema...

[sachouche]

Github user sachouche commented on the issue:

https://github.com/apache/drill/pull/1170
  
@amansinha100 can you please review this pull request?

Thanks!


---

[GitHub] drill pull request #1170: DRILL-6223: Fixed several Drillbit failures due to...

[sachouche]

GitHub user sachouche opened a pull request:

https://github.com/apache/drill/pull/1170

DRILL-6223: Fixed several Drillbit failures due to schema changes

Fixed several Issues due to Schema changes:
1) Changes in complex data types
Drill Query Failing when selecting all columns from a Complex Nested Data 
File (Parquet) Set). There are differences in Schema among the files:

The Parquet files exhibit differences both at the first level and within 
nested data types
A select * will not cause an exception but using a limit clause will
Note also this issue seems to happen only when multiple Drillbit minor 
fragments are involved (concurrency higher than one)

2) Dangling columns (both simple and complex)
This situation can be easily reproduced for:
- Select STAR queries which involve input data with different schemas
- LIMIT or / and PROJECT operators are used
- The data will be read from more than one minor fragment
- This is because individual readers have logic to handle such use-cases 
but not downstream operators
- So is reader-1 sends one batch with F1, F2, and F3
- The reader-2 sends batch F2, F3
- Then the LIMIT and PROJECT operator will fail to cleanup the dangling 
column F1 which will cause failures when downstream operators copy logic 
attempts copy the stale column F1
- This pull request adds logic to detect and eliminate dangling columns   

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sachouche/drill DRILL-6223

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1170.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1170


commit d986b6c7588c107bb7e49d2fc8eb3f25a60e1214
Author: Salim Achouche 
Date:   2018-02-21T02:17:14Z

DRILL-6223: Fixed several Drillbit failures due to schema changes




---

Re: [VOTE] Apache Drill release 1.13.0 - RC0

[Sorabh Hamirwasia]

  *   Downloaded binary tarball from [2] and deployed 2 node cluster
  *   Ran some basic queries using sqlline and Web UI with and without security 
enabled
  *   Verified user to bit secure connections using Plain/Kerberos
  *   Verified bit to bit secure connections using Kerberos.
 *   During testing I found an issue that local control message is still 
creating a connection, but is not related to this release. The issue is a 
regression because of state introduced in Drillbit endpoint as part of shutdown 
feature in 1.12. I have opened 
DRILL-6255 for this issue 
with details in it.
  *   Verified SPNEGO and FORM authentication for Web UI
  *   Ran and verify queries against sys.connections table
  *   Built C++ client on linux box using the source tarball from [2] and ran 
queries to secure and unsecure Drillbit


LGTM +1 (non-binding)


Thanks,
Sorabh


From: Vitalii Diravka 
Sent: Thursday, March 15, 2018 11:12 AM
To: dev@drill.apache.org
Subject: Re: [VOTE] Apache Drill release 1.13.0 - RC0

* Downloaded sources tarball from [2].
Ran drill in embedded mode on local debian machine. Ran tpch queries with
joins, group by, order by, limit, order by with limit statements.
Looked through logs - looks good.
* Build drill for [4] with MapR profile. Ran driillbit in distributed mode
on centos VM with MapR core. Ran queries for Hive 1.2, ran queries for Hive
2.1 (transactional and non-transactional tables). Connected to this
drillbit from remote machine via JDBC with java programm and with SQuirrel
using different drivers (prebuild "drill-jdbc-all-1.12.0.jar" from [2]
tarball, the driver after build [4] sources with default and MapR profiles
too) and ran a simple query. Ran the same with enabled custom
authentication (jdbc driver which is build with MapR profile works good
too).
In the process of testing jdbc connection I found the issue - DRILL-6251.
It is a regression. I have described the case in Jira. But I suppose isn't
critical for current Drill release.
* All unit test were passed for [4]. Total time on my machine was: 42:10 min

+1 (non-binding)

Kind regards
Vitalii

On Thu, Mar 15, 2018 at 4:49 PM, Vova Vysotskyi  wrote:

> - Downloaded source tar at [2], ran unit tests and all tests are passed.
> - Downloaded built tar at [2], submitted several TPCH queries from UI,
> checked that profiles are displayed correctly.
> - Connected from SQuirrel, ran several queries; ran queries from a java
> application, no issues were found.
>
> +1 (non-binding)
>
>
> 2018-03-15 13:19 GMT+02:00 Arina Yelchiyeva :
>
> >  - Built from the source [4] on Linux, run unit test.
> > - Downloaded the binary tarball [2], untarred and ran Drill in embedded
> > mode on Windows.
> > - Ran sample queries, checked system tables, profiles on Web UI, also
> logs
> > and index page.
> > - Created persistent and temporary tables, loaded custom UDFs.
> >
> > +1 (binding)
> >
> > Kind regards
> > Arina
> >
> > On Thu, Mar 15, 2018 at 1:39 AM, Aman Sinha 
> wrote:
> >
> > > - Downloaded the source tarball from [2] on my Linux VM, built and ran
> > the
> > > unit tests successfully
> > > - Downloaded the binary tarball onto my Macbook, untarred and ran Drill
> > in
> > > embedded mode
> > > - Ran several queries  against a TPC-DS SF1 data set, including CTAS
> > > statements with PARTITION BY and ran a few partition pruning queries
> > > - Tested query cancellation by cancelling a query that was taking long
> > time
> > > due to expanding join
> > > - Examined the run-time query profiles of these queries with and
> without
> > > parallelism.
> > > - Checked the maven artifacts on [3].
> > >
> > >  - Found one reference to JDK 7 : README.md says 'JDK 7' in the
> > > Prerequisites.  Ideally, this should be changed to JDK 8
> > >
> > > Overall, LGTM  +1 (binding)
> > >
> > >
> > > On Tue, Mar 13, 2018 at 3:58 AM, Parth Chandra 
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'd like to propose the first release candidate (RC0) of Apache
> Drill,
> > > > version 1.13.0.
> > > >
> > > > The release candidate covers a total of 113 resolved JIRAs [1].
> Thanks
> > > > to everyone
> > > > who contributed to this release.
> > > >
> > > > The tarball artifacts are hosted at [2] and the maven artifacts are
> > > hosted
> > > > at
> > > > [3].
> > > >
> > > > This release candidate is based on commit
> > > > cac2882d5a9e22fbc251e4caf622fe30242ad557 located at [4].
> > > >
> > > > Please download and try out the release.
> > > >
> > > > The vote ends at 1:00 PM UTC (5:00 AM PDT, 2:00 PM EET, 5:30 PM IST),
> > Mar
> > > > 16th, 2018
> > > >
> > > > [ ] +1
> > > > [ ] +0
> > > > [ ] -1
> > > >
> > > > Here's my vote: +1
> > > >
> > > >
> > > > [1 ]
> > > > 

[jira] [Created] (DRILL-6255) Drillbit while sending control message to itself creates a connection instead of submitting locally

[Sorabh Hamirwasia (JIRA)]

Sorabh Hamirwasia created DRILL-6255:


 Summary: Drillbit while sending control message to itself creates 
a connection instead of submitting locally
 Key: DRILL-6255
 URL: https://issues.apache.org/jira/browse/DRILL-6255
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.12.0
Reporter: Sorabh Hamirwasia
Assignee: Sorabh Hamirwasia
 Fix For: 1.14.0


With the new shutdown feature introduced in 1.12, there is a state introduced 
in DrillbitEndpoint. Due to this the equality check happening 
[here|https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/FragmentsRunner.java#L256]
 will result in false and hence the fragments supposed to be scheduled on 
Foreman will be treated as remote fragments and a connection will be created to 
schedule it. The equality check is false because localEndpoint state is STARTUP 
whereas state in assigned Drillbit is ONLINE.

I guess now we should update the equality check to verify just for address and 
control port to be same between assigned and local Drillbit endpoint. A test 
can be added for this based on _sys.memory_ table since that guarantees 
scheduling minor fragments on each Drillbit node.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (DRILL-6254) IllegalArgumentException: the requested size must be non-negative

[Khurram Faraaz (JIRA)]

Khurram Faraaz created DRILL-6254:
-

 Summary: IllegalArgumentException: the requested size must be 
non-negative
 Key: DRILL-6254
 URL: https://issues.apache.org/jira/browse/DRILL-6254
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.13.0
Reporter: Khurram Faraaz
Assignee: Padma Penumarthy
 Attachments: genAllTypesJSN.py

Flatten query fails due to IllegalArgumentException: the requested size must be 
non-negative.

Script to generate JSON data file is attached here.

{noformat}
0: jdbc:drill:schema=dfs.tmp> CREATE TABLE tbl_all_types_jsn_to_parquet AS 
. . . . . . . . . . . . . . > SELECT
. . . . . . . . . . . . . . > CAST( col_int AS INT) col_int, 
. . . . . . . . . . . . . . > CAST( col_bigint AS BIGINT) col_bigint, 
. . . . . . . . . . . . . . > CAST( col_char AS CHAR(10)) col_char, 
. . . . . . . . . . . . . . > CAST( col_fxdln_str AS VARCHAR(256)) 
col_fxdln_str, 
. . . . . . . . . . . . . . > CAST( col_varln_str AS VARCHAR(256)) 
col_varln_str, 
. . . . . . . . . . . . . . > CAST( col_float AS FLOAT) col_float, 
. . . . . . . . . . . . . . > CAST( col_double AS DOUBLE PRECISION) col_double, 
. . . . . . . . . . . . . . > CAST( col_date AS DATE) col_date, 
. . . . . . . . . . . . . . > CAST( col_time AS TIME) col_time, 
. . . . . . . . . . . . . . > CAST( col_tmstmp AS TIMESTAMP) col_tmstmp, 
. . . . . . . . . . . . . . > CAST( col_boolean AS BOOLEAN) col_boolean, 
. . . . . . . . . . . . . . > col_binary, 
. . . . . . . . . . . . . . > array_of_ints from `all_supported_types.json`;
+---++
| Fragment | Number of records written |
+---++
| 0_0 | 9 |
+---++
1 row selected (0.29 seconds)
{noformat}

Reset all options and set slice_target=1
alter system reset all;
alter system set `planner.slice_target`=1;

output_batch_size was set to its default value
drill.exec.memory.operator.output_batch_size = 16777216

 

{noformat}
select *, flatten(array_of_ints) from tbl_all_types_jsn_to_parquet;

Error: SYSTEM ERROR: IllegalArgumentException: the requested size must be 
non-negative

Fragment 0:0

[Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010] 
(state=,code=0)
0: jdbc:drill:schema=dfs.tmp>
{noformat}

Stack trace from drillbit.log

{noformat}
2018-03-15 12:19:43,916 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO 
o.a.drill.exec.work.foreman.Foreman - Query text for query id 
255538af-bcd5-98ee-32e0-68d98fc4a6fa: select *, flatten(array_of_ints) from 
tbl_all_types_jsn_to_parquet
2018-03-15 12:19:43,952 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO 
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2018-03-15 12:19:43,953 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO 
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, 
numFiles: 1
2018-03-15 12:19:43,966 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO 
o.a.d.exec.store.parquet.Metadata - Took 0 ms to get file statuses
2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO 
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 
using 1 threads. Time: 2ms total, 2.927366ms avg, 2ms max.
2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO 
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 
using 1 threads. Earliest start: 2.829000 μs, Latest start: 2.829000 μs, 
Average start: 2.829000 μs .
2018-03-15 12:19:43,969 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:foreman] INFO 
o.a.d.exec.store.parquet.Metadata - Took 3 ms to read file metadata
2018-03-15 12:19:44,000 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO 
o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2018-03-15 12:19:44,000 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO 
o.a.d.e.w.f.FragmentStatusReporter - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: 
State to report: RUNNING
2018-03-15 12:19:44,905 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO 
o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: 
State change requested RUNNING --> FAILED
2018-03-15 12:19:44,927 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] INFO 
o.a.d.e.w.fragment.FragmentExecutor - 255538af-bcd5-98ee-32e0-68d98fc4a6fa:0:0: 
State change requested FAILED --> FINISHED
2018-03-15 12:19:44,928 [255538af-bcd5-98ee-32e0-68d98fc4a6fa:frag:0:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalArgumentException: 
the requested size must be non-negative

Fragment 0:0

[Error Id: 480bae96-ae89-45a7-b937-011c0f87c14d on qa102-45.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 

HashAgg Unit Testing Design Doc

[Timothy Farkas]

Hi All,

I'm sharing a design doc to do some refactoring for HashAgg to make it easier 
to unit test, and reduce the dev / QA time required to make changes to it. 
Please take a look and leave comments.

https://docs.google.com/document/d/110BAWg3QXMfdmuqB0p3HuaoKpPGY-lqCRtHFxdh53Ds/edit#

I'll be updating https://issues.apache.org/jira/browse/DRILL-6253 to hold all 
the subtasks involved for this.

Thanks,
Tim

[jira] [Created] (DRILL-6253) HashAgg Unit Testing And Refactoring

[Timothy Farkas (JIRA)]

Timothy Farkas created DRILL-6253:
-

 Summary: HashAgg Unit Testing And Refactoring
 Key: DRILL-6253
 URL: https://issues.apache.org/jira/browse/DRILL-6253
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Timothy Farkas
Assignee: Timothy Farkas


This is a parent issue to hold all the subtasks required to refactor HashAgg to 
make it unit testable. Design doc

https://docs.google.com/document/d/110BAWg3QXMfdmuqB0p3HuaoKpPGY-lqCRtHFxdh53Ds/edit?usp=sharing





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] drill issue #1169: DRILL-6243: Added alert box to confirm shutdown of drillb...

[dvjyothsna]

Github user dvjyothsna commented on the issue:

https://github.com/apache/drill/pull/1169
  
@sohami Please review


---

[GitHub] drill pull request #1169: DRILL-6243: Added alert box to confirm shutdown of...

[dvjyothsna]

GitHub user dvjyothsna opened a pull request:

https://github.com/apache/drill/pull/1169

DRILL-6243: Added alert box to confirm shutdown of drillbit after cli…

…cking shutdown button

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dvjyothsna/drill DRILL-6243

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1169.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1169


commit 9b4b0700937e9d155324ebbd119171588dcf97c1
Author: dvjyothsna 
Date:   2018-03-14T23:36:42Z

DRILL-6243: Added alert box to confirm shutdown of drillbit after clicking 
shutdown button




---

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Pritesh Maker]

Congratulations, Vova!

On 3/15/18, 10:16 AM, "Arina Ielchiieva"  wrote:

 The Project Management Committee (PMC) for Apache Drill has
invited Volodymyr Vysotskyi to become a committer, and we are pleased to
announce that he has accepted.

Volodymyr has been contributing for Drill over a year. He contributed in
different areas, including code generation, json processing, function
implementations.
Also he actively participated in Calcite rebase and showed profound 
knowledge
in planning area.
Currently he is working on decimal's enhancement in Drill.

Congratulations Volodymyr and thank you for your contributions!

- Arina
(on behalf of the Apache Drill PMC)

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Vlad Rozov]

Congrats Vova!

Thank you,

Vlad

On 3/15/18 11:38, Gautam Parai wrote:

Congratulations Volodymyr!


Gautam


From: Timothy Farkas 
Sent: Thursday, March 15, 2018 11:10:40 AM
To: dev@drill.apache.org
Subject: Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

Congratulations Volodymyr!


From: Vitalii Diravka 
Sent: Thursday, March 15, 2018 10:32:34 AM
To: dev@drill.apache.org
Subject: Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

Congrats Vova! Well deserved!

Kind regards
Vitalii

On Thu, Mar 15, 2018 at 7:26 PM, Khurram Faraaz  wrote:


Congratulations Volodymyr!


From: Arina Ielchiieva 
Sent: Thursday, March 15, 2018 10:16:51 AM
To: dev@drill.apache.org
Subject: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

The Project Management Committee (PMC) for Apache Drill has
invited Volodymyr Vysotskyi to become a committer, and we are pleased to
announce that he has accepted.

Volodymyr has been contributing for Drill over a year. He contributed in
different areas, including code generation, json processing, function
implementations.
Also he actively participated in Calcite rebase and showed profound
knowledge
in planning area.
Currently he is working on decimal's enhancement in Drill.

Congratulations Volodymyr and thank you for your contributions!

- Arina
(on behalf of the Apache Drill PMC)

[GitHub] drill issue #1168: DRILL-6246: Reduced the size of the jdbc-all jar file

[sohami]

Github user sohami commented on the issue:

https://github.com/apache/drill/pull/1168
  
+1 LGTM. Thanks for making the changes.


---

[GitHub] drill issue #1159: DRILL-6215: Changed Statement to PreparedStatement in Jdb...

[kkhatua]

Github user kkhatua commented on the issue:

https://github.com/apache/drill/pull/1159
  
@kfaraaz Are there unit tests specific to the JDBC Storage Plugin? They're 
not the same as the JDBC unit tests, which test the Drill JDBC driver.

While the change seems straightforward, I'm not sure if all JDBC drivers 
support PreparedStatement. For e.g., within Drill, the PreparedStatement is 
executed as a standard Statement object, which is why this _works_ functionally.

What happens if a JDBC driver backing the JDBC storage plugin does not 
support PreparedStatement (i.e. it is a No-Op)? 


---

[jira] [Created] (DRILL-6252) Foreman node is going down when the non foreman node is stopped

[Venkata Jyothsna Donapati (JIRA)]

Venkata Jyothsna Donapati created DRILL-6252:


 Summary: Foreman node is going down when the non foreman node is 
stopped
 Key: DRILL-6252
 URL: https://issues.apache.org/jira/browse/DRILL-6252
 Project: Apache Drill
  Issue Type: Bug
Reporter: Venkata Jyothsna Donapati


Two drillbits are running. I'm running a join query over parquet and tried to 
stop the non-foreman node using drillbit.sh stop. The query fails with *"Error: 
DATA_READ ERROR: Exception occurred while reading from disk".* The non-foreman 
node goes down. The foreman node also goes down. When I looked at the 
drillbit.log of both foreman and non-foreman I found that there is memory leak  
"Memory was leaked by query. Memory leaked: 
(2097152)\nAllocator(op:2:0:0:HashPartitionSender) 
100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are 
the stack traces for memory leaks 

 

[Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]

org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)

Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 
(res/actual/peak/limit)

 

 

Fragment 2:1 

[Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]

        at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
 ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]

        at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
 [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]

        at 
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
 [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]

        at 
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
 [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]

        at 
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) 
[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]

        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[na:1.8.0_161]

        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[na:1.8.0_161]

        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]

Caused by: java.lang.IllegalStateException: Memory was leaked by query. Memory 
leaked: (3145728)

Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 
(res/actual/peak/limit)

 

Ping me for the logs and more information.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Gautam Parai]

Congratulations Volodymyr!


Gautam


From: Timothy Farkas 
Sent: Thursday, March 15, 2018 11:10:40 AM
To: dev@drill.apache.org
Subject: Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

Congratulations Volodymyr!


From: Vitalii Diravka 
Sent: Thursday, March 15, 2018 10:32:34 AM
To: dev@drill.apache.org
Subject: Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

Congrats Vova! Well deserved!

Kind regards
Vitalii

On Thu, Mar 15, 2018 at 7:26 PM, Khurram Faraaz  wrote:

> Congratulations Volodymyr!
>
> 
> From: Arina Ielchiieva 
> Sent: Thursday, March 15, 2018 10:16:51 AM
> To: dev@drill.apache.org
> Subject: [ANNOUNCE] New Committer: Volodymyr Vysotskyi
>
> The Project Management Committee (PMC) for Apache Drill has
> invited Volodymyr Vysotskyi to become a committer, and we are pleased to
> announce that he has accepted.
>
> Volodymyr has been contributing for Drill over a year. He contributed in
> different areas, including code generation, json processing, function
> implementations.
> Also he actively participated in Calcite rebase and showed profound
> knowledge
> in planning area.
> Currently he is working on decimal's enhancement in Drill.
>
> Congratulations Volodymyr and thank you for your contributions!
>
> - Arina
> (on behalf of the Apache Drill PMC)
>

[GitHub] drill pull request #1168: DRILL-6246: Reduced the size of the jdbc-all jar f...

[sachouche]

Github user sachouche commented on a diff in the pull request:

https://github.com/apache/drill/pull/1168#discussion_r174889287
  
--- Diff: exec/jdbc-all/pom.xml ---
@@ -473,6 +473,8 @@
org/yaml/**
hello/**
webapps/**
+   **/org/apache/calcite/avatica/metrics/**
+   **/org/apache/calcite/avatica/org/**
--- End diff --

done!


---

[GitHub] drill pull request #1168: DRILL-6246: Reduced the size of the jdbc-all jar f...

[sohami]

Github user sohami commented on a diff in the pull request:

https://github.com/apache/drill/pull/1168#discussion_r174882554
  
--- Diff: exec/jdbc-all/pom.xml ---
@@ -473,6 +473,8 @@
org/yaml/**
hello/**
webapps/**
+   **/org/apache/calcite/avatica/metrics/**
+   **/org/apache/calcite/avatica/org/**
--- End diff --

Would be great to make that change. Other than that, changes looks good to 
me


---

Re: [VOTE] Apache Drill release 1.13.0 - RC0

[Vitalii Diravka]

* Downloaded sources tarball from [2].
Ran drill in embedded mode on local debian machine. Ran tpch queries with
joins, group by, order by, limit, order by with limit statements.
Looked through logs - looks good.
* Build drill for [4] with MapR profile. Ran driillbit in distributed mode
on centos VM with MapR core. Ran queries for Hive 1.2, ran queries for Hive
2.1 (transactional and non-transactional tables). Connected to this
drillbit from remote machine via JDBC with java programm and with SQuirrel
using different drivers (prebuild "drill-jdbc-all-1.12.0.jar" from [2]
tarball, the driver after build [4] sources with default and MapR profiles
too) and ran a simple query. Ran the same with enabled custom
authentication (jdbc driver which is build with MapR profile works good
too).
In the process of testing jdbc connection I found the issue - DRILL-6251.
It is a regression. I have described the case in Jira. But I suppose isn't
critical for current Drill release.
* All unit test were passed for [4]. Total time on my machine was: 42:10 min

+1 (non-binding)

Kind regards
Vitalii

On Thu, Mar 15, 2018 at 4:49 PM, Vova Vysotskyi  wrote:

> - Downloaded source tar at [2], ran unit tests and all tests are passed.
> - Downloaded built tar at [2], submitted several TPCH queries from UI,
> checked that profiles are displayed correctly.
> - Connected from SQuirrel, ran several queries; ran queries from a java
> application, no issues were found.
>
> +1 (non-binding)
>
>
> 2018-03-15 13:19 GMT+02:00 Arina Yelchiyeva :
>
> >  - Built from the source [4] on Linux, run unit test.
> > - Downloaded the binary tarball [2], untarred and ran Drill in embedded
> > mode on Windows.
> > - Ran sample queries, checked system tables, profiles on Web UI, also
> logs
> > and index page.
> > - Created persistent and temporary tables, loaded custom UDFs.
> >
> > +1 (binding)
> >
> > Kind regards
> > Arina
> >
> > On Thu, Mar 15, 2018 at 1:39 AM, Aman Sinha 
> wrote:
> >
> > > - Downloaded the source tarball from [2] on my Linux VM, built and ran
> > the
> > > unit tests successfully
> > > - Downloaded the binary tarball onto my Macbook, untarred and ran Drill
> > in
> > > embedded mode
> > > - Ran several queries  against a TPC-DS SF1 data set, including CTAS
> > > statements with PARTITION BY and ran a few partition pruning queries
> > > - Tested query cancellation by cancelling a query that was taking long
> > time
> > > due to expanding join
> > > - Examined the run-time query profiles of these queries with and
> without
> > > parallelism.
> > > - Checked the maven artifacts on [3].
> > >
> > >  - Found one reference to JDK 7 : README.md says 'JDK 7' in the
> > > Prerequisites.  Ideally, this should be changed to JDK 8
> > >
> > > Overall, LGTM  +1 (binding)
> > >
> > >
> > > On Tue, Mar 13, 2018 at 3:58 AM, Parth Chandra 
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'd like to propose the first release candidate (RC0) of Apache
> Drill,
> > > > version 1.13.0.
> > > >
> > > > The release candidate covers a total of 113 resolved JIRAs [1].
> Thanks
> > > > to everyone
> > > > who contributed to this release.
> > > >
> > > > The tarball artifacts are hosted at [2] and the maven artifacts are
> > > hosted
> > > > at
> > > > [3].
> > > >
> > > > This release candidate is based on commit
> > > > cac2882d5a9e22fbc251e4caf622fe30242ad557 located at [4].
> > > >
> > > > Please download and try out the release.
> > > >
> > > > The vote ends at 1:00 PM UTC (5:00 AM PDT, 2:00 PM EET, 5:30 PM IST),
> > Mar
> > > > 16th, 2018
> > > >
> > > > [ ] +1
> > > > [ ] +0
> > > > [ ] -1
> > > >
> > > > Here's my vote: +1
> > > >
> > > >
> > > > [1 ]
> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > > > projectId=12313820=12342096
> > > >
> > > > [2] http://home.apache.org/~parthc/drill/releases/1.13.0/rc0/
> > > >
> > > > [3] https://repository.apache.org/content/repositories/
> > > orgapachedrill-1046
> > > >
> > > > [4] https://github.com/parthchandra/drill/tree/drill-1.13.0
> > > >
> > >
> >
>
>
>
> --
> Kind regards,
> Volodymyr Vysotskyi
>

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Timothy Farkas]

Congratulations Volodymyr!


From: Vitalii Diravka 
Sent: Thursday, March 15, 2018 10:32:34 AM
To: dev@drill.apache.org
Subject: Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

Congrats Vova! Well deserved!

Kind regards
Vitalii

On Thu, Mar 15, 2018 at 7:26 PM, Khurram Faraaz  wrote:

> Congratulations Volodymyr!
>
> 
> From: Arina Ielchiieva 
> Sent: Thursday, March 15, 2018 10:16:51 AM
> To: dev@drill.apache.org
> Subject: [ANNOUNCE] New Committer: Volodymyr Vysotskyi
>
> The Project Management Committee (PMC) for Apache Drill has
> invited Volodymyr Vysotskyi to become a committer, and we are pleased to
> announce that he has accepted.
>
> Volodymyr has been contributing for Drill over a year. He contributed in
> different areas, including code generation, json processing, function
> implementations.
> Also he actively participated in Calcite rebase and showed profound
> knowledge
> in planning area.
> Currently he is working on decimal's enhancement in Drill.
>
> Congratulations Volodymyr and thank you for your contributions!
>
> - Arina
> (on behalf of the Apache Drill PMC)
>

[GitHub] drill pull request #1168: DRILL-6246: Reduced the size of the jdbc-all jar f...

[sachouche]

Github user sachouche commented on a diff in the pull request:

https://github.com/apache/drill/pull/1168#discussion_r174870229
  
--- Diff: exec/jdbc-all/pom.xml ---
@@ -473,6 +473,8 @@
org/yaml/**
hello/**
webapps/**
+   **/org/apache/calcite/avatica/metrics/**
+   **/org/apache/calcite/avatica/org/**
--- End diff --

The Apache jar size has shrunk by ~1MB from over 34MB to 33393541 bytes; I 
guess we can decrease the limit from 35MB to 34MB.


---

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Vitalii Diravka]

Congrats Vova! Well deserved!

Kind regards
Vitalii

On Thu, Mar 15, 2018 at 7:26 PM, Khurram Faraaz  wrote:

> Congratulations Volodymyr!
>
> 
> From: Arina Ielchiieva 
> Sent: Thursday, March 15, 2018 10:16:51 AM
> To: dev@drill.apache.org
> Subject: [ANNOUNCE] New Committer: Volodymyr Vysotskyi
>
> The Project Management Committee (PMC) for Apache Drill has
> invited Volodymyr Vysotskyi to become a committer, and we are pleased to
> announce that he has accepted.
>
> Volodymyr has been contributing for Drill over a year. He contributed in
> different areas, including code generation, json processing, function
> implementations.
> Also he actively participated in Calcite rebase and showed profound
> knowledge
> in planning area.
> Currently he is working on decimal's enhancement in Drill.
>
> Congratulations Volodymyr and thank you for your contributions!
>
> - Arina
> (on behalf of the Apache Drill PMC)
>

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Kunal Khatua]

Congratulations, Volodymyr !

On Thu, Mar 15, 2018 at 10:26 AM, Khurram Faraaz  wrote:

> Congratulations Volodymyr!
>
> 
> From: Arina Ielchiieva 
> Sent: Thursday, March 15, 2018 10:16:51 AM
> To: dev@drill.apache.org
> Subject: [ANNOUNCE] New Committer: Volodymyr Vysotskyi
>
> The Project Management Committee (PMC) for Apache Drill has
> invited Volodymyr Vysotskyi to become a committer, and we are pleased to
> announce that he has accepted.
>
> Volodymyr has been contributing for Drill over a year. He contributed in
> different areas, including code generation, json processing, function
> implementations.
> Also he actively participated in Calcite rebase and showed profound
> knowledge
> in planning area.
> Currently he is working on decimal's enhancement in Drill.
>
> Congratulations Volodymyr and thank you for your contributions!
>
> - Arina
> (on behalf of the Apache Drill PMC)
>

[GitHub] drill pull request #1168: DRILL-6246: Reduced the size of the jdbc-all jar f...

[sohami]

Github user sohami commented on a diff in the pull request:

https://github.com/apache/drill/pull/1168#discussion_r174866472
  
--- Diff: exec/jdbc-all/pom.xml ---
@@ -473,6 +473,8 @@
org/yaml/**
hello/**
webapps/**
+   **/org/apache/calcite/avatica/metrics/**
+   **/org/apache/calcite/avatica/org/**
--- End diff --

After removing these exclusion for apache profile what is the size of 
jdbc-all jar ? based on final size we should reduce the allowed maxSize from 
~35MB to that value.


---

Re: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Khurram Faraaz]

Congratulations Volodymyr!


From: Arina Ielchiieva 
Sent: Thursday, March 15, 2018 10:16:51 AM
To: dev@drill.apache.org
Subject: [ANNOUNCE] New Committer: Volodymyr Vysotskyi

The Project Management Committee (PMC) for Apache Drill has
invited Volodymyr Vysotskyi to become a committer, and we are pleased to
announce that he has accepted.

Volodymyr has been contributing for Drill over a year. He contributed in
different areas, including code generation, json processing, function
implementations.
Also he actively participated in Calcite rebase and showed profound knowledge
in planning area.
Currently he is working on decimal's enhancement in Drill.

Congratulations Volodymyr and thank you for your contributions!

- Arina
(on behalf of the Apache Drill PMC)

[jira] [Created] (DRILL-6251) Queries from system tables are hang

[Vitalii Diravka (JIRA)]

Vitalii Diravka created DRILL-6251:
--

 Summary: Queries from system tables are hang
 Key: DRILL-6251
 URL: https://issues.apache.org/jira/browse/DRILL-6251
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.13.0
Reporter: Vitalii Diravka


On CentoOS cluster Drill hangs while querying sys tables after "use dfs;" 
(embedded or distributed mode):
{code}
0: jdbc:drill:> select * from sys.version;
+--+---+--++++
| version  | commit_id |
commit_message|commit_time |build_email 
| build_time |
+--+---+--++++
| 1.13.0   | 796fcf051b3553c4597abbdca5ca247b139734ba  | [maven-release-plugin] 
prepare release drill-1.13.0  | 13.03.2018 @ 11:39:14 IST  | par...@apache.org  
| 13.03.2018 @ 13:13:45 IST  |
+--+---+--++++
1 row selected (3.784 seconds)
0: jdbc:drill:> use dfs;
+---+--+
|  ok   | summary  |
+---+--+
| true  | Default schema changed to [dfs]  |
+---+--+
1 row selected (0.328 seconds)
0: jdbc:drill:> select * from sys.version;
Error: Statement canceled (state=,code=0)
0: jdbc:drill:> 
{code}
*Note*: there is not failure on local debian machine in embedded mode.
dfs pugin configs are default (with "connection": "file:///", other file 
systems works good).

This failure is connected to DRILL-5089 and calcite rebase. 
Related commits:
https://github.com/apache/drill/commit/450e67094eb6e9a6484d7f86c49b51c77a08d7b2
https://github.com/apache/drill/commit/18a71a38f6bd1fd33d21d1c68fc23c5901b0080a

After analyzing in remote debug I found the following flow:
"dfs" DynamicRootSchema is created, then a new "sys" one is created.
After calcite validate "sys" SimpleCalciteSchema is created. But in 
WorkspaceSchemaFactory#create  wrong WorkspaceConfig is left and "/" is 
combined with "sys".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[ANNOUNCE] New Committer: Volodymyr Vysotskyi

[Arina Ielchiieva]

 The Project Management Committee (PMC) for Apache Drill has
invited Volodymyr Vysotskyi to become a committer, and we are pleased to
announce that he has accepted.

Volodymyr has been contributing for Drill over a year. He contributed in
different areas, including code generation, json processing, function
implementations.
Also he actively participated in Calcite rebase and showed profound knowledge
in planning area.
Currently he is working on decimal's enhancement in Drill.

Congratulations Volodymyr and thank you for your contributions!

- Arina
(on behalf of the Apache Drill PMC)

[GitHub] drill issue #1168: DRILL-6246: Reduced the size of the jdbc-all jar file

[sachouche]

Github user sachouche commented on the issue:

https://github.com/apache/drill/pull/1168
  
@sohami can you please review this pull request?

Thanks!


---

[GitHub] drill pull request #1168: DRILL-6246: Reduced the size of the jdbc-all jar f...

[sachouche]

GitHub user sachouche opened a pull request:

https://github.com/apache/drill/pull/1168

DRILL-6246: Reduced the size of the jdbc-all jar file

- The jdbc-all client jar has been growing in size from version to the next 
(~20MB in version 1.10, ~27MB in version 1.12, and 34MB in version 1.13
- Note the exact size of the size depends on the maven profile used during 
compilation (e.g., "mapr" which have more excludes)
- Originally, I tried to exclude the following calcite/avatica packages 
(responsible for the recent size increase): metrics, proto, org/apache/, 
com/fasterxml, com/google but some tests failed as the calcite code was 
referencing some of them
- In this pull request, I have excluded the following packages: 
calcite/avatica/org and calcite/avatica/metrics

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sachouche/drill DRILL-6246

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1168.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1168


commit 6b6188a6c1677595c479f67fc2e33af091c36818
Author: Salim Achouche 
Date:   2018-03-14T02:16:06Z

DRILL-6246: Reduced the size of the jdbc-all jar file




---

Re: [VOTE] Apache Drill release 1.13.0 - RC0

[Vova Vysotskyi]

- Downloaded source tar at [2], ran unit tests and all tests are passed.
- Downloaded built tar at [2], submitted several TPCH queries from UI,
checked that profiles are displayed correctly.
- Connected from SQuirrel, ran several queries; ran queries from a java
application, no issues were found.

+1 (non-binding)


2018-03-15 13:19 GMT+02:00 Arina Yelchiyeva :

>  - Built from the source [4] on Linux, run unit test.
> - Downloaded the binary tarball [2], untarred and ran Drill in embedded
> mode on Windows.
> - Ran sample queries, checked system tables, profiles on Web UI, also logs
> and index page.
> - Created persistent and temporary tables, loaded custom UDFs.
>
> +1 (binding)
>
> Kind regards
> Arina
>
> On Thu, Mar 15, 2018 at 1:39 AM, Aman Sinha  wrote:
>
> > - Downloaded the source tarball from [2] on my Linux VM, built and ran
> the
> > unit tests successfully
> > - Downloaded the binary tarball onto my Macbook, untarred and ran Drill
> in
> > embedded mode
> > - Ran several queries  against a TPC-DS SF1 data set, including CTAS
> > statements with PARTITION BY and ran a few partition pruning queries
> > - Tested query cancellation by cancelling a query that was taking long
> time
> > due to expanding join
> > - Examined the run-time query profiles of these queries with and without
> > parallelism.
> > - Checked the maven artifacts on [3].
> >
> >  - Found one reference to JDK 7 : README.md says 'JDK 7' in the
> > Prerequisites.  Ideally, this should be changed to JDK 8
> >
> > Overall, LGTM  +1 (binding)
> >
> >
> > On Tue, Mar 13, 2018 at 3:58 AM, Parth Chandra 
> wrote:
> >
> > > Hi all,
> > >
> > > I'd like to propose the first release candidate (RC0) of Apache Drill,
> > > version 1.13.0.
> > >
> > > The release candidate covers a total of 113 resolved JIRAs [1]. Thanks
> > > to everyone
> > > who contributed to this release.
> > >
> > > The tarball artifacts are hosted at [2] and the maven artifacts are
> > hosted
> > > at
> > > [3].
> > >
> > > This release candidate is based on commit
> > > cac2882d5a9e22fbc251e4caf622fe30242ad557 located at [4].
> > >
> > > Please download and try out the release.
> > >
> > > The vote ends at 1:00 PM UTC (5:00 AM PDT, 2:00 PM EET, 5:30 PM IST),
> Mar
> > > 16th, 2018
> > >
> > > [ ] +1
> > > [ ] +0
> > > [ ] -1
> > >
> > > Here's my vote: +1
> > >
> > >
> > > [1 ]
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > > projectId=12313820=12342096
> > >
> > > [2] http://home.apache.org/~parthc/drill/releases/1.13.0/rc0/
> > >
> > > [3] https://repository.apache.org/content/repositories/
> > orgapachedrill-1046
> > >
> > > [4] https://github.com/parthchandra/drill/tree/drill-1.13.0
> > >
> >
>



-- 
Kind regards,
Volodymyr Vysotskyi

Re: [VOTE] Apache Drill release 1.13.0 - RC0

[Arina Yelchiyeva]

 - Built from the source [4] on Linux, run unit test.
- Downloaded the binary tarball [2], untarred and ran Drill in embedded
mode on Windows.
- Ran sample queries, checked system tables, profiles on Web UI, also logs
and index page.
- Created persistent and temporary tables, loaded custom UDFs.

+1 (binding)

Kind regards
Arina

On Thu, Mar 15, 2018 at 1:39 AM, Aman Sinha  wrote:

> - Downloaded the source tarball from [2] on my Linux VM, built and ran the
> unit tests successfully
> - Downloaded the binary tarball onto my Macbook, untarred and ran Drill in
> embedded mode
> - Ran several queries  against a TPC-DS SF1 data set, including CTAS
> statements with PARTITION BY and ran a few partition pruning queries
> - Tested query cancellation by cancelling a query that was taking long time
> due to expanding join
> - Examined the run-time query profiles of these queries with and without
> parallelism.
> - Checked the maven artifacts on [3].
>
>  - Found one reference to JDK 7 : README.md says 'JDK 7' in the
> Prerequisites.  Ideally, this should be changed to JDK 8
>
> Overall, LGTM  +1 (binding)
>
>
> On Tue, Mar 13, 2018 at 3:58 AM, Parth Chandra  wrote:
>
> > Hi all,
> >
> > I'd like to propose the first release candidate (RC0) of Apache Drill,
> > version 1.13.0.
> >
> > The release candidate covers a total of 113 resolved JIRAs [1]. Thanks
> > to everyone
> > who contributed to this release.
> >
> > The tarball artifacts are hosted at [2] and the maven artifacts are
> hosted
> > at
> > [3].
> >
> > This release candidate is based on commit
> > cac2882d5a9e22fbc251e4caf622fe30242ad557 located at [4].
> >
> > Please download and try out the release.
> >
> > The vote ends at 1:00 PM UTC (5:00 AM PDT, 2:00 PM EET, 5:30 PM IST), Mar
> > 16th, 2018
> >
> > [ ] +1
> > [ ] +0
> > [ ] -1
> >
> > Here's my vote: +1
> >
> >
> > [1 ]
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12313820=12342096
> >
> > [2] http://home.apache.org/~parthc/drill/releases/1.13.0/rc0/
> >
> > [3] https://repository.apache.org/content/repositories/
> orgapachedrill-1046
> >
> > [4] https://github.com/parthchandra/drill/tree/drill-1.13.0
> >
>

[GitHub] drill issue #1166: DRILL-6016 - Fix for Error reading INT96 created by Apach...

[rajrahul]

Github user rajrahul commented on the issue:

https://github.com/apache/drill/pull/1166
  
@parthchandra @vdiravka I have added the test case using the same parquet 
file(2.9k bytes). I tried creating a smaller file using Spark, but could not 
replicate the behavior. I have rebased the changes on the same commit and PR.


---

[jira] [Created] (DRILL-6250) Sqlline start command with password appears in the sqlline.log

[Anton Gozhiy (JIRA)]

Anton Gozhiy created DRILL-6250:
---

 Summary: Sqlline start command with password appears in the 
sqlline.log
 Key: DRILL-6250
 URL: https://issues.apache.org/jira/browse/DRILL-6250
 Project: Apache Drill
  Issue Type: Bug
Affects Versions: 1.13.0
Reporter: Anton Gozhiy


*Prerequisites:*
 *1.* Log level is set to "all" in the conf/logback.xml:
{code:xml}




{code}
*2.* PLAIN authentication mechanism is configured:
{code:java}
  security.user.auth: {
enabled: true,
packages += "org.apache.drill.exec.rpc.user.security",
impl: "pam",
pam_profiles: [ "sudo", "login" ]
  }
{code}
*Steps:*
 *1.* Start the drillbits
 *2.* Connect by sqlline:
{noformat}
/opt/mapr/drill/drill-1.13.0/bin/sqlline -u "jdbc:drill:zk=node1:5181;" -n 
user1 -p 1234
{noformat}
*3.* Use check the sqlline logs:
{noformat}
tail -F log/sqlline.log|grep 1234 -a5 -b5
{noformat}
*Expected result:* Logs shouldn't contain clear-text passwords

*Actual result:* The logs contain the sqlline start command with password:
{noformat}
# system properties
35333-"java" : {
35352-# system properties
35384:"command" : "sqlline.SqlLine -d org.apache.drill.jdbc.Driver 
--maxWidth=1 --color=true -u jdbc:drill:zk=node1:5181; -n user1 -p 1234",
35535-# system properties
35567-"launcher" : "SUN_STANDARD"
35607-}
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)