Re: [NOTICE] Maven 3.6.3

[Arina Ielchiieva]

Looks like decision was made to stay on the Maven 3.6.3.
For those who want to follow the discussion, please see Jira and PR:
https://issues.apache.org/jira/browse/DRILL-7708 
<https://issues.apache.org/jira/browse/DRILL-7708>
https://github.com/apache/drill/pull/2061 
<https://github.com/apache/drill/pull/2061>

Kind regards,
Arina

> On Apr 17, 2020, at 8:59 PM, Paul Rogers  wrote:
> 
> Hi Arina,
> 
> Thanks for keeping us up to date!
> 
> As it turns out, I use Ubuntu (Linux Mint) for development. Maven is 
> installed as a package using apt-get. Packages can lag behind a bit. The 
> latest maven available via apt-get is 3.6.0.
> 
> It is a nuisance to install a new version outside the package manager. I 
> changed the Maven version in the root pom.xml to 3.6.0 and the build seemed 
> to work. Any reason we need the absolute latest version rather than just 
> 3.6.0 or later?
> 
> The workaround for now is to manually edit the pom.xml file on each checkout, 
> then revert the change before commit. Can we maybe adjust the "official" 
> version instead?
> 
> 
> Thanks,
> - Paul
> 
> 
> 
>On Friday, April 17, 2020, 5:09:49 AM PDT, Arina Ielchiieva 
>  wrote:  
> 
> Hi all,
> 
> Starting from Drill 1.18.0 (and current master from commit 20ad3c9 [1]), 
> Drill build will require Maven 3.6.3, otherwise build will fail.
> Please make sure you have Maven 3.6.3 installed on your environments. 
> 
> [1] 
> https://github.com/apache/drill/commit/20ad3c9837e9ada149c246fc7a4ac1fe02de6fe8
> 
> Kind regards,
> Arina  

[NOTICE] Maven 3.6.3

[Arina Ielchiieva]

Hi all,

Starting from Drill 1.18.0 (and current master from commit 20ad3c9 [1]), Drill 
build will require Maven 3.6.3, otherwise build will fail.
Please make sure you have Maven 3.6.3 installed on your environments. 

[1] 
https://github.com/apache/drill/commit/20ad3c9837e9ada149c246fc7a4ac1fe02de6fe8

Kind regards,
Arina

Re: scaling drill in an openshift (K8s) cluster

[Arina Ielchiieva]

Please see https://issues.apache.org/jira/browse/DRILL-7563 
, maybe it will be helpful.

Kind regards,
Arina

> On Mar 24, 2020, at 6:04 PM, Dobes Vandermeer  wrote:
> 
> I was able to get drill up and running inside a k8s cluster but I didn't 
> connect to it from outside the cluster, so the DNS names were always 
> resolvable by the client(s).
> 
> I had to run it as a statefulset to ensure the DNS names are stable, 
> otherwise the drillbits couldn't talk to each other, either.
> 
> On 3/24/2020 6:37:44 AM, Jaimes, Rafael - 0993 - MITLL 
>  wrote:
> I’m seeing a problem with scaling the number of pod instances in the 
> replication controller because they aren’t reporting their hostnames 
> properly. This was a common problem that got fixed in scalable architectures 
> like ZooKeeper and Kafka (see reference at bottom I think this was related).
>  
> In Drill’s case, ZooKeeper is able to see all of the drillbits, however, the 
> hostnames are only locally addressable within the cluster, so as soon as you 
> perform a query it fails since the client can’t find the drillbit that it got 
> assigned, its hostname isn’t externally addressable.
>  
> Kafka fixes this by allowing an override for advertised names. Has anyone 
> gotten Drill to scale in a K8s cluster?
>  
> https://issues.apache.org/jira/browse/KAFKA-1070

[ANNOUNCE] New PMC member: Bohdan Kazydub

[Arina Ielchiieva]

I am pleased to announce that Drill PMC invited Bohdan Kazydub to the PMC and
he has accepted the invitation.

Congratulations Bohdan and welcome!

- Arina
(on behalf of Drill PMC)

[ANNOUNCE] New Committer: Denys Ordynskiy

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Denys
Ordynskiy to become a committer, and we are pleased to announce that he has
accepted.

Denys has been contributing to Drill for more than a year. He did many
contributions as a QA, he found, tested and verified important bugs and
features. Recently he has actively participated in Hadoop 3 migration
verification and actively tested current and previous releases. He also
contributed into drill-test-framework to automate Drill tests.

Welcome Denys, and thank you for your contributions!

- Arina
(on behalf of Drill PMC)

[ANNOUNCE] New PMC Chair of Apache Drill

[Arina Ielchiieva]

Hi all,

It has been a honor to serve as Drill Chair during the past year but it's
high time for the new one...

I am very pleased to announce that the Drill PMC has voted to elect Charles
Givre as the new PMC chair of Apache Drill. He has also been approved
unanimously by the Apache Board in last board meeting.

Congratulations, Charles!

Kind regards,
Arina

Re: [ANNOUNCE] New Committer: Anton Gozhyi

[Arina Ielchiieva]

Congratulations Anton! Thanks for your contributions.

Kind regards,
Arina

On Mon, Jul 29, 2019 at 12:55 PM Павел Семенов 
wrote:

> Congratulations Anton ! Well done.
>
> пн, 29 июл. 2019 г. в 12:54, Bohdan Kazydub :
>
> > Congratulations Anton!
> >
> > On Mon, Jul 29, 2019 at 12:44 PM Igor Guzenko <
> ihor.huzenko@gmail.com>
> > wrote:
> >
> > > Congratulations Anton!
> > >
> > > On Mon, Jul 29, 2019 at 12:09 PM denysord88 
> > wrote:
> > >
> > > > Congratulations Anton! Well deserved!
> > > >
> > > > On 07/29/2019 12:02 PM, Volodymyr Vysotskyi wrote:
> > > > > The Project Management Committee (PMC) for Apache Drill has invited
> > > Anton
> > > > > Gozhyi to become a committer, and we are pleased to announce that
> he
> > > has
> > > > > accepted.
> > > > >
> > > > > Anton Gozhyi has been contributing to Drill for more than a year
> and
> > a
> > > > > half. He did significant contributions as a QA, including reporting
> > > > > non-trivial issues and working on automation of Drill tests. All
> the
> > > > issues
> > > > > reported by Anton have a clear description of the problem, steps to
> > > > > reproduce and expected behavior. Besides contributions as a QA,
> Anton
> > > > made
> > > > > high-quality fixes into Drill.
> > > > >
> > > > > Welcome Anton, and thank you for your contributions!
> > > > >
> > > > > - Volodymyr
> > > > > (on behalf of Drill PMC)
> > > > >
> > > >
> > > >
> > >
> >
>
>
> --
>
> *Kind regards,*
> *Pavel Semenov*
>

[ANNOUNCE] New Committer: Igor Guzenko

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Igor
Guzenko to become a committer, and we are pleased to announce that he has
accepted.

Igor has been contributing into Drill for 9 months and made a number of
significant contributions, including cross join syntax support, Hive views
support, as well as improving performance for Hive show schema and unit
tests. Currently he is working on supporting Hive complex types
[DRILL-3290]. He already added support for list type and working on struct
and canonical map.

Welcome Igor, and thank you for your contributions!

- Arina
(on behalf of the Apache Drill PMC)

[ANNOUNCE] New Committer: Bohdan Kazydub

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Bohdan
Kazydub to become a committer, and we are pleased to announce that he has
accepted.

Bohdan has been contributing into Drill for more than a year. His
contributions include
logging and various functions handling improvements, planning optimizations
and S3 improvements / fixes. His recent work includes Calcite 1.19 / 1.20
[DRILL-7200] and implementation of canonical Map [DRILL-7096].

Welcome Bohdan, and thank you for your contributions!

- Arina
(on behalf of the Apache Drill PMC)

Re: May Apache Drill board report

[Arina Ielchiieva]

Thanks everybody for the feedback, report has been posted.

Kind regards,
Arina

On Sat, May 4, 2019 at 2:29 AM SorabhApache  wrote:

> +1
>
> On Fri, May 3, 2019 at 4:22 PM Boaz Ben-Zvi  wrote:
>
> > No comments; looks fine; +1
> >
> > On 5/3/19 3:10 PM, Aman Sinha wrote:
> > > +1
> > >
> > > On Fri, May 3, 2019 at 1:40 PM Volodymyr Vysotskyi <
> volody...@apache.org
> > >
> > > wrote:
> > >
> > >> Looks good, +1
> > >>
> > >>
> > >> Пт, 3 трав. 2019 23:32 користувач Arina Ielchiieva 
> > >> пише:
> > >>
> > >>> Hi all,
> > >>>
> > >>> please take a look at the draft board report for the last quarter and
> > let
> > >>> me know if you have any comments.
> > >>>
> > >>> Thanks,
> > >>> Arina
> > >>>
> > >>> =
> > >>>
> > >>> ## Description:
> > >>> - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
> > >>>Storage.
> > >>>
> > >>> ## Issues:
> > >>>   - There are no issues requiring board attention at this time.
> > >>>
> > >>> ## Activity:
> > >>> - Since the last board report, Drill has released version 1.16.0,
> > >> including
> > >>>the following enhancements:
> > >>>- CREATE OR REPLACE SCHEMA command to define a schema for text
> files
> > >>>- REFRESH TABLE METADATA command can generate metadata cache files
> > for
> > >>>specific columns
> > >>>- ANALYZE TABLE statement to computes statistics on Parquet data
> > >>>- SYSLOG (RFC-5424) Format Plugin
> > >>>- NEAREST DATE function to facilitate time series analysis
> > >>>- Format plugin for LTSV files
> > >>>- Ability to query Hive views
> > >>>- Upgrade to SQLLine 1.7
> > >>>- Apache Calcite upgrade to 1.18.0
> > >>>- Several Drill Web UI improvements, including:
> > >>>   - Storage plugin management improvements
> > >>>   - Query progress indicators and warnings
> > >>>   - Ability to limit the result size for better UI response
> > >>>   - Ability to sort the list of profiles in the Drill Web UI
> > >>>   - Display query state in query result page
> > >>>   - Button to reset the options filter
> > >>>
> > >>> - Drill User Meetup will be held on May 22, 2019. Two talks are
> > planned:
> > >>>- Alibaba's Usage of Apache Drill for querying a Time Series
> > Database
> > >>>- What’s new with Apache Drill 1.16 & a demo of Schema
> Provisioning
> > >>>
> > >>> ## Health report:
> > >>> - The project is healthy. Development activity as reflected in the
> pull
> > >>>requests and JIRAs is good.
> > >>> - Activity on the dev and user mailing lists are stable.
> > >>> - One PMC member was added in the last period.
> > >>>
> > >>> ## PMC changes:
> > >>>
> > >>> - Currently 24 PMC members.
> > >>> - Sorabh Hamirwasia was added to the PMC on Fri Apr 05 2019
> > >>>
> > >>> ## Committer base changes:
> > >>>
> > >>> - Currently 51 committers.
> > >>> - No new committers added in the last 3 months
> > >>> - Last committer addition was Salim Achouche at Mon Dec 17 2018
> > >>>
> > >>> ## Releases:
> > >>>
> > >>> - 1.16.0 was released on Thu May 02 2019
> > >>>
> > >>> ## Mailing list activity:
> > >>>
> > >>> - d...@drill.apache.org:
> > >>> - 406 subscribers (down -10 in the last 3 months):
> > >>> - 2299 emails sent to list (1903 in previous quarter)
> > >>>
> > >>> - iss...@drill.apache.org:
> > >>> - 17 subscribers (down -1 in the last 3 months):
> > >>> - 2373 emails sent to list (2233 in previous quarter)
> > >>>
> > >>> - user@drill.apache.org:
> > >>> - 582 subscribers (down -15 in the last 3 months):
> > >>> - 235 emails sent to list (227 in previous quarter)
> > >>>
> > >>> ## JIRA activity:
> > >>>
> > >>> - 214 JIRA tickets created in the last 3 months
> > >>> - 212 JIRA tickets closed/resolved in the last 3 months
> > >>>
> >
>

May Apache Drill board report

[Arina Ielchiieva]

Hi all,

please take a look at the draft board report for the last quarter and let
me know if you have any comments.

Thanks,
Arina

=

## Description:
- Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
  Storage.

## Issues:
 - There are no issues requiring board attention at this time.

## Activity:
- Since the last board report, Drill has released version 1.16.0, including
  the following enhancements:
  - CREATE OR REPLACE SCHEMA command to define a schema for text files
  - REFRESH TABLE METADATA command can generate metadata cache files for
  specific columns
  - ANALYZE TABLE statement to computes statistics on Parquet data
  - SYSLOG (RFC-5424) Format Plugin
  - NEAREST DATE function to facilitate time series analysis
  - Format plugin for LTSV files
  - Ability to query Hive views
  - Upgrade to SQLLine 1.7
  - Apache Calcite upgrade to 1.18.0
  - Several Drill Web UI improvements, including:
 - Storage plugin management improvements
 - Query progress indicators and warnings
 - Ability to limit the result size for better UI response
 - Ability to sort the list of profiles in the Drill Web UI
 - Display query state in query result page
 - Button to reset the options filter

- Drill User Meetup will be held on May 22, 2019. Two talks are planned:
  - Alibaba's Usage of Apache Drill for querying a Time Series Database
  - What’s new with Apache Drill 1.16 & a demo of Schema Provisioning

## Health report:
- The project is healthy. Development activity as reflected in the pull
  requests and JIRAs is good.
- Activity on the dev and user mailing lists are stable.
- One PMC member was added in the last period.

## PMC changes:

- Currently 24 PMC members.
- Sorabh Hamirwasia was added to the PMC on Fri Apr 05 2019

## Committer base changes:

- Currently 51 committers.
- No new committers added in the last 3 months
- Last committer addition was Salim Achouche at Mon Dec 17 2018

## Releases:

- 1.16.0 was released on Thu May 02 2019

## Mailing list activity:

- d...@drill.apache.org:
   - 406 subscribers (down -10 in the last 3 months):
   - 2299 emails sent to list (1903 in previous quarter)

- iss...@drill.apache.org:
   - 17 subscribers (down -1 in the last 3 months):
   - 2373 emails sent to list (2233 in previous quarter)

- user@drill.apache.org:
   - 582 subscribers (down -15 in the last 3 months):
   - 235 emails sent to list (227 in previous quarter)

## JIRA activity:

- 214 JIRA tickets created in the last 3 months
- 212 JIRA tickets closed/resolved in the last 3 months

Re: [VOTE] Apache Drill Release 1.16.0 - RC1

[Arina Ielchiieva]

Taking into account previous emails, looks like we'll need to have new RC.
I also suggest to include https://issues.apache.org/jira/browse/DRILL-7201 into
new RC.

Kind regards,
Arina

On Wed, Apr 24, 2019 at 2:44 PM Volodymyr Vysotskyi 
wrote:

> Also, I have noticed that for the prebuilt tar, the following query on my
> machine returns the wrong results:
>
> apache drill> select * from sys.version;
>
> +-+--++---+---+---+
> | version |commit_id |
>commit_message |commit_time|
> build_email|build_time |
>
> +-+--++---+---+---+
> | 1.16.0  | b3db1ff4b0d29210593c4485125578cca7a64b42 | DRILL-7188: Revert
> DRILL-6642: Update protocol-buffers version | 21.04.2019 @ 15:35:28 PDT |
> sor...@apache.org | 22.04.2019 @ 09:07:35 PDT |
>
> +-+--++---+---+---+
> 1 row selected (1.318 seconds)
>
> The root cause for this problem is that drill-format-mapr-1.16.0.jar jar
> contains git.properties file with incorrect version, and this file was the
> first one which was found by the class loader.
>
> I think this is a blocker for the release.
>
> Kind regards,
> Volodymyr Vysotskyi
>
>
> On Wed, Apr 24, 2019 at 2:31 PM Anton Gozhiy  wrote:
>
> > Clarification to my last message:
> > I downloaded Drill from here:
> >
> >
> http://home.apache.org/~sorabh/drill/releases/1.16.0/rc1/apache-drill-1.16.0-src.tar.gz
> > and built it by command:
> > mvn clean install -DskipTests
> >
> > On Wed, Apr 24, 2019 at 1:53 PM Anton Gozhiy 
> wrote:
> >
> > > Hi All,
> > >
> > > I found an issue with Drill version, used the provided rc1 source:
> > > apache drill> select * from sys.version;
> > >
> > >
> >
> +-+---++-+-++
> > > | version | commit_id | commit_message | commit_time | build_email |
> > > build_time |
> > >
> > >
> >
> +-+---++-+-++
> > > | 1.16.0  | Unknown   || | Unknown |
> > >   |
> > >
> > >
> >
> +-+---++-+-++
> > >
> > > Although there is a valid git.properties file in the Drill root
> > directory:
> > > #Generated by Git-Commit-Id-Plugin
> > > #Mon Apr 22 09:52:07 PDT 2019
> > > git.branch=cf5b758e0a4c22b75bfb02ac2653ff09415ddf53
> > > git.build.host=SHamirw-E755.local
> > > git.build.time=22.04.2019 @ 09\:52\:07 PDT
> > > git.build.user.email=sor...@apache.org
> > > git.build.user.name=Sorabh Hamirwasia
> > > git.build.version=1.16.0
> > > git.closest.tag.commit.count=0
> > > git.closest.tag.name=drill-1.16.0
> > > git.commit.id=cf5b758e0a4c22b75bfb02ac2653ff09415ddf53
> > > git.commit.id.abbrev=cf5b758
> > > git.commit.id.describe=drill-1.16.0-0-gcf5b758
> > > git.commit.id.describe-short=drill-1.16.0-0
> > > git.commit.message.full=[maven-release-plugin] prepare release
> > drill-1.16.0
> > > git.commit.message.short=[maven-release-plugin] prepare release
> > > drill-1.16.0
> > > git.commit.time=22.04.2019 @ 09\:08\:36 PDT
> > > git.commit.user.email=sor...@apache.org
> > > git.commit.user.name=Sorabh Hamirwasia
> > > git.dirty=false
> > > git.remote.origin.url=https\://github.com/apache/drill.git
> > > git.tags=drill-1.16.0
> > > git.total.commit.count=3568
> > >
> > > But looks like it doesn't get into the classpath.
> > > Could someone take a look into this?
> > >
> > > Thanks!
> > >
> > > On Wed, Apr 24, 2019 at 11:50 AM Volodymyr Vysotskyi <
> > volody...@apache.org>
> > > wrote:
> > >
> > >> Hi Sorabh,
> > >>
> > >> Sorry for being picky, but looks like the key you have published was
> > >> generated for non-apache email: sohami.apa...@gmail.com. According to
> > the
> > >> [1], it is highly recommended to use Apache email address as the
> primary
> > >> User-ID.
> > >>
> > >> [1] https://www.apache.org/dev/release-signing#user-id
> > >>
> > >> Kind regards,
> > >> Volodymyr Vysotskyi
> > >>
> > >>
> > >> On Wed, Apr 24, 2019 at 10:10 AM Jyothsna Reddy <
> jyothsna@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Built it from cloning the git branch and unit tests on my Linux VM
> > (time
> > >> > taken - 43 min).
> > >> > Tested new features of metadata caching by creating v4 cache files
> > using
> > >> > new Refresh Metadata commands and manually verified the cache files.
> > >> Tried
> > >> > a few queries that use metadata cache and verified results.
> > 

[ANNOUNCE] New PMC member: Sorabh Hamirwasia

[Arina Ielchiieva]

I am pleased to announce that Drill PMC invited Sorabh Hamirwasia to
the PMC and
he has accepted the invitation.

Congratulations Sorabh and welcome!

- Arina
(on behalf of Drill PMC)

Re: January Apache Drill board report

[Arina Ielchiieva]

Report has been submitted.

Kind regards,
Arina

On Fri, Feb 1, 2019 at 3:40 PM Arina Ielchiieva  wrote:

> Thanks everybody for the feedback. Please see below updated report:
>
> =
>
> ## Description:
>  - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
> Storage.
>
> ## Issues:
>  - There are no issues requiring board attention at this time.
>
> ## Activity:
>  - Since the last board report, Drill has released version 1.15.0,
>including the following enhancements:
>- Add capability to do index based planning and execution
>- CROSS join support
>- INFORMATION_SCHEMA FILES and FUNCTIONS were added
>- Support for TIMESTAMPADD and TIMESTAMPDIFF functions
>- Ability to secure znodes with custom ACLs
>- Upgrade to SQLLine 1.6
>- Parquet filter pushdown for VARCHAR and DECIMAL data types
>- Support JPPD (Join Predicate Push Down)
>- Lateral join functionality was enabled by default
>- Multiple Web UI improvements to simplify the use of options and
> submit queries
>- Query performance with the semi-join functionality was improved
>- Support for aliases in the GROUP BY clause
>- Option to prevent Drill from returning a result set for DDL
> statements
>- Storage plugin names became case-insensitive
>
> - Drill Developer Day was held on November 14, 2018: a variety of
>   technical design issues were discussed, including Apache Arrow
> integration,
>   Metadata and Resource management, Storage plugins, etc.
>
> - Drill User Meetup was held on November 14, 2018: use cases of Drill and
>   indexing support were presented.
>
> ## Health report:
>  - The project is healthy. Development activity
>as reflected in the pull requests and JIRAs is good.
>  - Activity on the dev and user mailing lists are stable.
>  - Three committers were added in the last period.
>
> ## PMC changes:
>
>  - Currently 23 PMC members.
>  - No new PMC members added in the last 3 months
>  - Last PMC addition was Charles Givre on Mon Sep 03 2018
>
> ## Committer base changes:
>
>  - Currently 51 committers.
>  - New commmitters:
> - Hanumath Rao Maduri was added as a committer on Thu Nov 01 2018
> - Karthikeyan Manivannan was added as a committer on Fri Dec 07 2018
> - Salim Achouche was added as a committer on Mon Dec 17 2018
>
> ## Releases:
>
>  - 1.15.0 was released on Mon Dec 31 2018
>
> ## Mailing list activity:
>
>  - d...@drill.apache.org:
> - 415 subscribers (down -12 in the last 3 months):
> - 2066 emails sent to list (2653 in previous quarter)
>
>  - iss...@drill.apache.org:
> - 18 subscribers (up 0 in the last 3 months):
> - 2480 emails sent to list (3228 in previous quarter)
>
>  - user@drill.apache.org:
> - 592 subscribers (down -5 in the last 3 months):
> - 249 emails sent to list (310 in previous quarter)
>
>
> ## JIRA activity:
>
>  - 196 JIRA tickets created in the last 3 months
>  - 171 JIRA tickets closed/resolved in the last 3 months
>
>
> On Fri, Feb 1, 2019 at 12:22 PM Bohdan Kazydub 
> wrote:
>
>> Hi, Arina
>>
>> Just to clarify, one of the options, the one "to return null for empty
>> string" (`drill.exec.functions.cast_empty_string_to_null`),
>> was present for some time (from Drill 0.8.0 at least) and affected CASTs
>> from string to numeric types only (if the option is set to true, empty
>> string in CAST is treated as NULL).
>> In 1.15 the option was expanded to affect CASTs to other types and TO_*
>> functions.
>>
>>
>> On Fri, Feb 1, 2019 at 9:24 AM Abhishek Girish 
>> wrote:
>>
>> > +1. Looks good!
>> >
>> > On Thu, Jan 31, 2019 at 9:15 AM Vitalii Diravka 
>> > wrote:
>> >
>> > > +1
>> > >
>> > > Kind regards
>> > > Vitalii
>> > >
>> > >
>> > > On Thu, Jan 31, 2019 at 6:18 PM Aman Sinha 
>> wrote:
>> > >
>> > > > Thanks for putting this together, Arina.
>> > > > The Drill Developer Day and Meetup were separate events, so you can
>> > split
>> > > > them up.
>> > > >   - A half day Drill Developer Day was held on Nov 14.  A variety of
>> > > > technical design issues were discussed.
>> > > >   - A Drill user meetup was held on the same evening.  2
>> presentations
>> > -
>> > > > one on use case for Drill and one about indexing support in Drill
>> were
>> > > > presented.
>> > > >
>> > > > Rest of the report LGTM.
>>

Re: January Apache Drill board report

[Arina Ielchiieva]

Thanks everybody for the feedback. Please see below updated report:

=

## Description:
 - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
Storage.

## Issues:
 - There are no issues requiring board attention at this time.

## Activity:
 - Since the last board report, Drill has released version 1.15.0,
   including the following enhancements:
   - Add capability to do index based planning and execution
   - CROSS join support
   - INFORMATION_SCHEMA FILES and FUNCTIONS were added
   - Support for TIMESTAMPADD and TIMESTAMPDIFF functions
   - Ability to secure znodes with custom ACLs
   - Upgrade to SQLLine 1.6
   - Parquet filter pushdown for VARCHAR and DECIMAL data types
   - Support JPPD (Join Predicate Push Down)
   - Lateral join functionality was enabled by default
   - Multiple Web UI improvements to simplify the use of options and submit
queries
   - Query performance with the semi-join functionality was improved
   - Support for aliases in the GROUP BY clause
   - Option to prevent Drill from returning a result set for DDL statements
   - Storage plugin names became case-insensitive

- Drill Developer Day was held on November 14, 2018: a variety of
  technical design issues were discussed, including Apache Arrow
integration,
  Metadata and Resource management, Storage plugins, etc.

- Drill User Meetup was held on November 14, 2018: use cases of Drill and
  indexing support were presented.

## Health report:
 - The project is healthy. Development activity
   as reflected in the pull requests and JIRAs is good.
 - Activity on the dev and user mailing lists are stable.
 - Three committers were added in the last period.

## PMC changes:

 - Currently 23 PMC members.
 - No new PMC members added in the last 3 months
 - Last PMC addition was Charles Givre on Mon Sep 03 2018

## Committer base changes:

 - Currently 51 committers.
 - New commmitters:
- Hanumath Rao Maduri was added as a committer on Thu Nov 01 2018
- Karthikeyan Manivannan was added as a committer on Fri Dec 07 2018
- Salim Achouche was added as a committer on Mon Dec 17 2018

## Releases:

 - 1.15.0 was released on Mon Dec 31 2018

## Mailing list activity:

 - d...@drill.apache.org:
- 415 subscribers (down -12 in the last 3 months):
- 2066 emails sent to list (2653 in previous quarter)

 - iss...@drill.apache.org:
- 18 subscribers (up 0 in the last 3 months):
- 2480 emails sent to list (3228 in previous quarter)

 - user@drill.apache.org:
- 592 subscribers (down -5 in the last 3 months):
- 249 emails sent to list (310 in previous quarter)


## JIRA activity:

 - 196 JIRA tickets created in the last 3 months
 - 171 JIRA tickets closed/resolved in the last 3 months


On Fri, Feb 1, 2019 at 12:22 PM Bohdan Kazydub 
wrote:

> Hi, Arina
>
> Just to clarify, one of the options, the one "to return null for empty
> string" (`drill.exec.functions.cast_empty_string_to_null`),
> was present for some time (from Drill 0.8.0 at least) and affected CASTs
> from string to numeric types only (if the option is set to true, empty
> string in CAST is treated as NULL).
> In 1.15 the option was expanded to affect CASTs to other types and TO_*
> functions.
>
>
> On Fri, Feb 1, 2019 at 9:24 AM Abhishek Girish  wrote:
>
> > +1. Looks good!
> >
> > On Thu, Jan 31, 2019 at 9:15 AM Vitalii Diravka 
> > wrote:
> >
> > > +1
> > >
> > > Kind regards
> > > Vitalii
> > >
> > >
> > > On Thu, Jan 31, 2019 at 6:18 PM Aman Sinha 
> wrote:
> > >
> > > > Thanks for putting this together, Arina.
> > > > The Drill Developer Day and Meetup were separate events, so you can
> > split
> > > > them up.
> > > >   - A half day Drill Developer Day was held on Nov 14.  A variety of
> > > > technical design issues were discussed.
> > > >   - A Drill user meetup was held on the same evening.  2
> presentations
> > -
> > > > one on use case for Drill and one about indexing support in Drill
> were
> > > > presented.
> > > >
> > > > Rest of the report LGTM.
> > > >
> > > > -Aman
> > > >
> > > >
> > > > On Thu, Jan 31, 2019 at 7:58 AM Arina Ielchiieva 
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > please take a look at the draft board report for the last quarter
> and
> > > let
> > > > > me know if you have any comments.
> > > > >
> > > > > Thanks,
> > > > > Arina
> > > > >
> > > > > =
> > > > >
> > > > > ## Description:
> > > > >  - Drill is a Schema-

January Apache Drill board report

[Arina Ielchiieva]

Hi all,

please take a look at the draft board report for the last quarter and let
me know if you have any comments.

Thanks,
Arina

=

## Description:
 - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
Storage.

## Issues:
 - There are no issues requiring board attention at this time.

## Activity:
 - Since the last board report, Drill has released version 1.15.0,
   including the following enhancements:
   - Add capability to do index based planning and execution
   - CROSS join support
   - INFORMATION_SCHEMA FILES and FUNCTIONS were added
   - Support for TIMESTAMPADD and TIMESTAMPDIFF functions
   - Ability to secure znodes with custom ACLs
   - Upgrade to SQLLine 1.6
   - Parquet filter pushdown for VARCHAR and DECIMAL data types
   - Support JPPD (Join Predicate Push Down)
   - Lateral join functionality was enabled by default
   - Multiple Web UI improvements to simplify the use of options and submit
queries
   - Query performance with the semi-join functionality was improved
   - Support for aliases in the GROUP BY clause
   - Options to return null for empty string and prevents Drill from
returning
 a result set for DDL statements
   - Storage plugin names became case-insensitive

- Drill developer meet up was held on November 14, 2018.

## Health report:
 - The project is healthy. Development activity
   as reflected in the pull requests and JIRAs is good.
 - Activity on the dev and user mailing lists are stable.
 - Three committers were added in the last period.

## PMC changes:

 - Currently 23 PMC members.
 - No new PMC members added in the last 3 months
 - Last PMC addition was Charles Givre on Mon Sep 03 2018

## Committer base changes:

 - Currently 51 committers.
 - New commmitters:
- Hanumath Rao Maduri was added as a committer on Thu Nov 01 2018
- Karthikeyan Manivannan was added as a committer on Fri Dec 07 2018
- Salim Achouche was added as a committer on Mon Dec 17 2018

## Releases:

 - 1.15.0 was released on Mon Dec 31 2018

## Mailing list activity:

 - d...@drill.apache.org:
- 415 subscribers (down -12 in the last 3 months):
- 2066 emails sent to list (2653 in previous quarter)

 - iss...@drill.apache.org:
- 18 subscribers (up 0 in the last 3 months):
- 2480 emails sent to list (3228 in previous quarter)

 - user@drill.apache.org:
- 592 subscribers (down -5 in the last 3 months):
- 249 emails sent to list (310 in previous quarter)


## JIRA activity:

 - 196 JIRA tickets created in the last 3 months
 - 171 JIRA tickets closed/resolved in the last 3 months

Schema Provision project

[Arina Ielchiieva]

Hi all,

here is a link [1] to the slides about Schema Provision project we
presented on the last hangout.
For more details please refer to DRILL-6835 [2].

[1]
https://docs.google.com/presentation/d/1Qki64_Q8rymUN4G4rNR7dADeRKhZhNS_sDcyGUj7Ca0/edit?usp=sharing
[2] https://issues.apache.org/jira/browse/DRILL-6835

Kind regards,
Arina

Re: [VOTE] Apache Drill release 1.15.0 - RC0

[Arina Ielchiieva]

I think above issues are blockers, Vitalii please cancel the vote

Kind regards,
Arina

On Wed, Dec 19, 2018 at 6:09 AM Aman Sinha  wrote:

> @vita...@apache.org   any idea why there's an
> extraneous directory in the source ?
> drwxrwxr-x vitalii/vitalii   0 2018-12-18 03:48
> apache-drill-1.15.0-src/${project.*basedir*}/
>
> drwxrwxr-x vitalii/vitalii   0 2018-12-18 03:48
> apache-drill-1.15.0-src/${project.*basedir*}/src/
>
> drwxrwxr-x vitalii/vitalii   0 2018-12-18 03:48
> apache-drill-1.15.0-src/${project.*basedir*}/src/site/
>
> drwxrwxr-x vitalii/vitalii   0 2018-12-18 03:48
> apache-drill-1.15.0-src/${project.*basedir*}/src/site/resources/
>
> drwxrwxr-x vitalii/vitalii   0 2018-12-18 03:48
> apache-drill-1.15.0-src/${project.*basedir*}/src/site/resources/repo/
>
> On Tue, Dec 18, 2018 at 10:58 AM Vitalii Diravka 
> wrote:
>
> > Hi all,
> >
> > I'd like to propose the first release candidate (RC0) of Apache Drill,
> > version 1.15.0.
> >
> > The release candidate covers a total of 185 resolved JIRAs [1].
> > Thanks to everyone who contributed to this release.
> >
> > The tarball artifacts are hosted at [2] and the maven artifacts are
> hosted
> > at [3].
> >
> > This release candidate is based on commit
> > ff797695de0e27a732c22e2410cbef58abbfcef3 located at [4].
> >
> > Please download and try out the release.
> >
> > The vote ends at 7:00 PM UTC (11:00 AM PDT, 9:00 PM EET, 0:30 AM IST),
> Dec
> > 21th, 2018
> >
> > [ ] +1
> > [ ] +0
> > [ ] -1
> >
> > I have found two issues, which are not observed earlier [5], [6]. Should
> we
> > consider them as blockers?
> >
> >   [1]
> > *
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12343317==12313820
> > <
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12343317==12313820
> > >*
> >
> >   [2] *http://home.apache.org/~vitalii/drill/releases/1.15.0/rc0/
> > *
> >
> >   [3] *
> > https://repository.apache.org/content/repositories/orgapachedrill-1058/
> >  >*
> >
> >   [4] *https://github.com/vdiravka/drill/commits/drill-1.15.0
> > *
> >
> >   [5] https://issues.apache.org/jira/browse/DRILL-6912
> >
> >   [6] https://issues.apache.org/jira/browse/DRILL-6913
> >
> > Kind regards
> > Vitalii
> >
>

[ANNOUNCE] New Committer: Salim Achouche

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Salim
Achouche to become a committer, and we are pleased to announce that he has
accepted.

Salim Achouche [1] started contributing to the Drill project in 2017. He
has made many improvements for the parquet reader, including performance
for flat data types, columnar parquet batch sizing functionality, fixed
various bugs and memory leaks. He also optimized implicit columns handling
with scanner and improved sql pattern contains performance.

Welcome Salim, and thank you for your contributions!

- Arina
(on behalf of Drill PMC)

[ANNOUNCE] New Committer: Karthikeyan Manivannan

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Karthikeyan
Manivannan to become a committer, and we are pleased to announce that he
has accepted.

Karthik started contributing to the Drill project in 2016. He has
implemented changes in various Drill areas, including batch sizing,
security, code-gen, C++ part. One of his latest improvements is  ACL
support for Drill ZK nodes.

Welcome Karthik, and thank you for your contributions!

- Arina
(on behalf of Drill PMC)

Re: November Apache Drill board report

[Arina Ielchiieva]

Thanks everyone for the feedback. Report has been submitted.

Kind regards,
Arina

On Wed, Nov 7, 2018 at 6:28 PM Arina Yelchiyeva 
wrote:

> Release notes were generated automatically based on resolved Jiras with
> fix version 1.14.0.
>
> Updated the report.
> I am going to sen it tomorrow if there won't be any other comments.
>
> --
>
> =
>
>  ## Description:
>  - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
> Storage.
>
> ## Issues:
>  - There are no issues requiring board attention at this time.
>
> ## Activity:
>  - Since the last board report, Drill has released version 1.14.0,
>including the following enhancements:
> - Drill in a Docker container
> - Image metadata format plugin
> - Upgrade to Calcite 1.16.0
> - Kafka plugin push down support
> - Phonetic and String functions
> - Enhanced decimal data support
> - Spill to disk for the Hash Join support
> - CGROUPs resource management support
> - Lateral / Unnest support (disabled by default)
> - Support Transitive Closure during Filter Push Down and Partition Pruning
> - Batch processing improvements to limit the amount of memory for Hash
> Join,
>   Union All, Project, Hash Aggregate and Nested Loop Join.
>  - There were active discussions about schema provision in Drill.
>Based on these discussions two projects are currently evolving:
>Drill metastore and schema provision in the file and in a query.
>  - Apache Drill book has been written by two PMC members (Charles and
> Paul).
>  - Drill developer meet up will be held on November 14, 2018.
>
>The following areas are going to be discussed:
> - Storage plugins
> - Schema discovery & Evolution
> - Metadata Management
> - Resource management
> - Integration with Apache Arrow
>
> ## Health report:
>  - The project is healthy. Development activity
>as reflected in the pull requests and JIRAs is good.
>  - Activity on the dev and user mailing lists are stable.
>  - Three committers and three new PMC member were added in the last period.
>
> ## PMC changes:
>
>  - Currently 23 PMC members.
>  - New PMC members:
> - Boaz Ben-Zvi was added to the PMC on Fri Aug 17 2018
> - Charles Givre was added to the PMC on Mon Sep 03 2018
> - Vova Vysotskyi was added to the PMC on Fri Aug 24 2018
>
> ## Committer base changes:
>
>  - Currently 48 committers.
>  - New commmitters:
> - Chunhui Shi was added as a committer on Thu Sep 27 2018
> - Gautam Parai was added as a committer on Mon Oct 22 2018
> - Weijie Tong was added as a committer on Fri Aug 31 2018
>
> ## Releases:
>
>  - 1.14.0 was released on Sat Aug 04 2018
>
> ## Mailing list activity:
>
>  - d...@drill.apache.org:
> - 427 subscribers (down -6 in the last 3 months):
> - 2827 emails sent to list (2126 in previous quarter)
>
>  - iss...@drill.apache.org:
> - 18 subscribers (down -1 in the last 3 months):
> - 3487 emails sent to list (4769 in previous quarter)
>
>  - user@drill.apache.org:
> - 597 subscribers (down -6 in the last 3 months):
> - 332 emails sent to list (346 in previous quarter)
>
>
> ## JIRA activity:
>
>  - 164 JIRA tickets created in the last 3 months
>  - 128 JIRA tickets closed/resolved in the last 3 months
>
>
>
> On Wed, Nov 7, 2018 at 5:37 PM Padma Penumarthy <
> penumarthy.pa...@gmail.com> wrote:
>
>> 1.14 release notes should mention batch sizing also.
>>
>> For the board report, can you please add the following :
>> Batch processing improvements to limit the amount of memory for Hash Join,
>> Union All, Project, Hash Aggregate and Nested Loop Join.
>>
>> Just FYI for everyone. Here is the link to the document which has details
>> about batch sizing work:
>>
>> https://docs.google.com/document/d/1Z-67Y_KNcbA2YYWCHEwf2PUEmXRPWSXsw-CHnXW_98Q/
>>
>> Thanks
>> Padma
>>
>>
>>
>>
>>
>> On Wed, Nov 7, 2018 at 5:56 AM Arina Ielchiieva  wrote:
>>
>> > Hi Padma,
>> >
>> > I can include mention about batch sizing but I am not sure what I should
>> > mention, quick search over release notes shows a couple of changes
>> related
>> > to batch sizing:
>> > https://drill.apache.org/docs/apache-drill-1-14-0-release-notes/
>> > Could you please propose what I should include?
>> >
>> > @PMCs and committers
>> > Only one PMC member has given +1 for the report. Could more folks please
>> > review the report?
>> >
>> > Kind regards,
>> > Arina
>> >
>> > On Fri,

Re: November Apache Drill board report

[Arina Ielchiieva]

Hi Padma,

I can include mention about batch sizing but I am not sure what I should
mention, quick search over release notes shows a couple of changes related
to batch sizing:
https://drill.apache.org/docs/apache-drill-1-14-0-release-notes/
Could you please propose what I should include?

@PMCs and committers
Only one PMC member has given +1 for the report. Could more folks please
review the report?

Kind regards,
Arina

On Fri, Nov 2, 2018 at 8:33 PM Padma Penumarthy 
wrote:

> Hi Arina,
>
> Can you add batch sizing (for bunch of operators and parquet reader) also ?
>
> Thanks
> Padma
>
>
> On Fri, Nov 2, 2018 at 2:55 AM Arina Ielchiieva  wrote:
>
> > Sure, let's mention.
> > Updated the report.
> >
> > =
> >
> >  ## Description:
> >  - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
> > Storage.
> >
> > ## Issues:
> >  - There are no issues requiring board attention at this time.
> >
> > ## Activity:
> >  - Since the last board report, Drill has released version 1.14.0,
> >including the following enhancements:
> > - Drill in a Docker container
> > - Image metadata format plugin
> > - Upgrade to Calcite 1.16.0
> > - Kafka plugin push down support
> > - Phonetic and String functions
> > - Enhanced decimal data support
> > - Spill to disk for the Hash Join support
> > - CGROUPs resource management support
> > - Lateral / Unnest support (disabled by default)
> >  - There were active discussions about schema provision in Drill.
> >Based on these discussions two projects are currently evolving:
> >Drill metastore and schema provision in the file and in a query.
> >  - Apache Drill book has been written by two PMC members (Charles and
> > Paul).
> >  - Drill developer meet up will be held on November 14, 2018.
> >
> >The following areas are going to be discussed:
> > - Storage plugins
> > - Schema discovery & Evolution
> > - Metadata Management
> > - Resource management
> > - Integration with Apache Arrow
> >
> > ## Health report:
> >  - The project is healthy. Development activity
> >as reflected in the pull requests and JIRAs is good.
> >  - Activity on the dev and user mailing lists are stable.
> >  - Three committers and three new PMC member were added in the last
> period.
> >
> > ## PMC changes:
> >
> >  - Currently 23 PMC members.
> >  - New PMC members:
> > - Boaz Ben-Zvi was added to the PMC on Fri Aug 17 2018
> > - Charles Givre was added to the PMC on Mon Sep 03 2018
> > - Vova Vysotskyi was added to the PMC on Fri Aug 24 2018
> >
> > ## Committer base changes:
> >
> >  - Currently 48 committers.
> >  - New commmitters:
> > - Chunhui Shi was added as a committer on Thu Sep 27 2018
> > - Gautam Parai was added as a committer on Mon Oct 22 2018
> > - Weijie Tong was added as a committer on Fri Aug 31 2018
> >
> > ## Releases:
> >
> >  - 1.14.0 was released on Sat Aug 04 2018
> >
> > ## Mailing list activity:
> >
> >  - d...@drill.apache.org:
> > - 427 subscribers (down -6 in the last 3 months):
> > - 2827 emails sent to list (2126 in previous quarter)
> >
> >  - iss...@drill.apache.org:
> > - 18 subscribers (down -1 in the last 3 months):
> > - 3487 emails sent to list (4769 in previous quarter)
> >
> >  - user@drill.apache.org:
> > - 597 subscribers (down -6 in the last 3 months):
> > - 332 emails sent to list (346 in previous quarter)
> >
> >
> > ## JIRA activity:
> >
> >  - 164 JIRA tickets created in the last 3 months
> >  - 128 JIRA tickets closed/resolved in the last 3 months
> >
> >
> >
> > On Fri, Nov 2, 2018 at 12:25 AM Sorabh Hamirwasia 
> > wrote:
> >
> > > Hi Arina,
> > > Lateral/Unnest feature was part of 1.14 though it was disabled by
> > default.
> > > Should we mention it as part of 1.14 enhancements in the report?
> > >
> > > Thanks,
> > > Sorabh
> > >
> > > On Thu, Nov 1, 2018 at 9:29 AM Arina Yelchiyeva <
> > > arina.yelchiy...@gmail.com>
> > > wrote:
> > >
> > > > Thanks, Aman!  Updated the report.
> > > > I went too far with 2019, luckily the meet up will be much earlier :)
> > > >
> > > > =
> > > >
> > > >  ## Description:
> > > >  - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and
> Cloud
> > > > Stora

Re: November Apache Drill board report

[Arina Ielchiieva]

> - Charles Givre was added to the PMC on Mon Sep 03 2018
> > - Vova Vysotskyi was added to the PMC on Fri Aug 24 2018
> >
> > ## Committer base changes:
> >
> >  - Currently 48 committers.
> >  - New commmitters:
> > - Chunhui Shi was added as a committer on Thu Sep 27 2018
> > - Gautam Parai was added as a committer on Mon Oct 22 2018
> > - Weijie Tong was added as a committer on Fri Aug 31 2018
> >
> > ## Releases:
> >
> >  - 1.14.0 was released on Sat Aug 04 2018
> >
> > ## Mailing list activity:
> >
> >  - d...@drill.apache.org:
> > - 427 subscribers (down -6 in the last 3 months):
> > - 2827 emails sent to list (2126 in previous quarter)
> >
> >  - iss...@drill.apache.org:
> > - 18 subscribers (down -1 in the last 3 months):
> > - 3487 emails sent to list (4769 in previous quarter)
> >
> >  - user@drill.apache.org:
> > - 597 subscribers (down -6 in the last 3 months):
> > - 332 emails sent to list (346 in previous quarter)
> >
> >
> > ## JIRA activity:
> >
> >  - 164 JIRA tickets created in the last 3 months
> >  - 128 JIRA tickets closed/resolved in the last 3 months
> >
> > On Thu, Nov 1, 2018 at 6:20 PM Aman Sinha  wrote:
> >
> > >Docket container  ==> 'Docker'
> > >November 14, 2019  ==>  2018  :)   (this is wrong in email that was
> > sent
> > > out)
> > >
> > > Rest LGTM.
> > >
> > > On Thu, Nov 1, 2018 at 6:42 AM Arina Ielchiieva 
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > please take a look at the draft board report for the last quarter and
> > let
> > > > me know if you have any comments.
> > > >
> > > > Thanks,
> > > > Arina
> > > >
> > > > =
> > > >
> > > >  ## Description:
> > > >  - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and
> Cloud
> > > > Storage.
> > > >
> > > > ## Issues:
> > > >  - There are no issues requiring board attention at this time.
> > > >
> > > > ## Activity:
> > > >  - Since the last board report, Drill has released version 1.14.0,
> > > >including the following enhancements:
> > > > - Drill in a Docket container
> > > > - Image metadata format plugin
> > > > - Upgrade to Calcite 1.16.0
> > > > - Kafka plugin push down support
> > > > - Phonetic and String functions
> > > > - Enhanced decimal data support
> > > > - Spill to disk for the Hash Join support
> > > > - CGROUPs resource management support
> > > >  - There were active discussions about schema provision in Drill.
> > > >Based on these discussions two projects are currently evolving:
> > > >Drill metastore and schema provision in the file and in a query.
> > > >  - Apache Drill book has been written by two PMC members (Charles and
> > > > Paul).
> > > >  - Drill developer meet up will be held on November 14, 2019.
> > > >The following areas are going to be discussed:
> > > > - Storage plugins
> > > > - Schema discovery & Evolution
> > > > - Metadata Management
> > > > - Resource management
> > > > - Integration with Apache Arrow
> > > >
> > > > ## Health report:
> > > >  - The project is healthy. Development activity
> > > >as reflected in the pull requests and JIRAs is good.
> > > >  - Activity on the dev and user mailing lists are stable.
> > > >  - Three committers and three new PMC member were added in the last
> > > period.
> > > >
> > > > ## PMC changes:
> > > >
> > > >  - Currently 23 PMC members.
> > > >  - New PMC members:
> > > > - Boaz Ben-Zvi was added to the PMC on Fri Aug 17 2018
> > > > - Charles Givre was added to the PMC on Mon Sep 03 2018
> > > > - Vova Vysotskyi was added to the PMC on Fri Aug 24 2018
> > > >
> > > > ## Committer base changes:
> > > >
> > > >  - Currently 48 committers.
> > > >  - New commmitters:
> > > > - Chunhui Shi was added as a committer on Thu Sep 27 2018
> > > > - Gautam Parai was added as a committer on Mon Oct 22 2018
> > > > - Weijie Tong was added as a committer on Fri Aug 31 2018
> > > >
> > > > ## Releases:
> > > >
> > > >  - 1.14.0 was released on Sat Aug 04 2018
> > > >
> > > > ## Mailing list activity:
> > > >
> > > >  - d...@drill.apache.org:
> > > > - 427 subscribers (down -6 in the last 3 months):
> > > > - 2827 emails sent to list (2126 in previous quarter)
> > > >
> > > >  - iss...@drill.apache.org:
> > > > - 18 subscribers (down -1 in the last 3 months):
> > > > - 3487 emails sent to list (4769 in previous quarter)
> > > >
> > > >  - user@drill.apache.org:
> > > > - 597 subscribers (down -6 in the last 3 months):
> > > > - 332 emails sent to list (346 in previous quarter)
> > > >
> > > >
> > > > ## JIRA activity:
> > > >
> > > >  - 164 JIRA tickets created in the last 3 months
> > > >  - 128 JIRA tickets closed/resolved in the last 3 months
> > > >
> > >
> >
>

November Apache Drill board report

[Arina Ielchiieva]

Hi all,

please take a look at the draft board report for the last quarter and let
me know if you have any comments.

Thanks,
Arina

=

 ## Description:
 - Drill is a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud
Storage.

## Issues:
 - There are no issues requiring board attention at this time.

## Activity:
 - Since the last board report, Drill has released version 1.14.0,
   including the following enhancements:
- Drill in a Docket container
- Image metadata format plugin
- Upgrade to Calcite 1.16.0
- Kafka plugin push down support
- Phonetic and String functions
- Enhanced decimal data support
- Spill to disk for the Hash Join support
- CGROUPs resource management support
 - There were active discussions about schema provision in Drill.
   Based on these discussions two projects are currently evolving:
   Drill metastore and schema provision in the file and in a query.
 - Apache Drill book has been written by two PMC members (Charles and Paul).
 - Drill developer meet up will be held on November 14, 2019.
   The following areas are going to be discussed:
- Storage plugins
- Schema discovery & Evolution
- Metadata Management
- Resource management
- Integration with Apache Arrow

## Health report:
 - The project is healthy. Development activity
   as reflected in the pull requests and JIRAs is good.
 - Activity on the dev and user mailing lists are stable.
 - Three committers and three new PMC member were added in the last period.

## PMC changes:

 - Currently 23 PMC members.
 - New PMC members:
- Boaz Ben-Zvi was added to the PMC on Fri Aug 17 2018
- Charles Givre was added to the PMC on Mon Sep 03 2018
- Vova Vysotskyi was added to the PMC on Fri Aug 24 2018

## Committer base changes:

 - Currently 48 committers.
 - New commmitters:
- Chunhui Shi was added as a committer on Thu Sep 27 2018
- Gautam Parai was added as a committer on Mon Oct 22 2018
- Weijie Tong was added as a committer on Fri Aug 31 2018

## Releases:

 - 1.14.0 was released on Sat Aug 04 2018

## Mailing list activity:

 - d...@drill.apache.org:
- 427 subscribers (down -6 in the last 3 months):
- 2827 emails sent to list (2126 in previous quarter)

 - iss...@drill.apache.org:
- 18 subscribers (down -1 in the last 3 months):
- 3487 emails sent to list (4769 in previous quarter)

 - user@drill.apache.org:
- 597 subscribers (down -6 in the last 3 months):
- 332 emails sent to list (346 in previous quarter)


## JIRA activity:

 - 164 JIRA tickets created in the last 3 months
 - 128 JIRA tickets closed/resolved in the last 3 months

[ANNOUNCE] New Committer: Hanumath Rao Maduri

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Hanumath
Rao Maduri to become a committer, and we are pleased to announce that he
has accepted.

Hanumath became a contributor in 2017, making changes mostly in the Drill
planning side, including lateral / unnest support. He is also one of the
contributors of index based planning and execution support.

Welcome Hanumath, and thank you for your contributions!

- Arina
(on behalf of Drill PMC)

[ANNOUNCE] New Committer: Gautam Parai

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Gautam
Parai to become a committer, and we are pleased to announce that he has
accepted.

Gautam has become a contributor since 2016, making changes in various Drill
areas including planning side. He is also one of the contributors of the
upcoming feature to support index based planning and execution.

Welcome Gautam, and thank you for your contributions!

- Arina
(on behalf of Drill PMC)

[ANNOUNCE] New Committer: Chunhui Shi

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Chunhui
Shi to become a committer, and we are pleased to announce that he has
accepted.

Chunhui Shi has become a contributor since 2016, making changes in various
Drill areas. He has shown profound knowledge in Drill planning side during
his work to support lateral join. He is also one of the contributors of the
upcoming feature to support index based planning and execution.

Welcome Chunhui, and thank you for your contributions!

- Arina
(on behalf of Drill PMC)

Re: [IDEAS] Drill start up quotes

[Arina Ielchiieva]

Thanks everybody for the ideas!
I have included at least one quote from each author (including those who
posted on  the Twitter).

Look for your quotes in the next Drill 1.15.0 release!

Kind regards,
Arina

On Thu, Sep 13, 2018 at 1:15 AM Chunhui Shi 
wrote:

> Some more quotes:
>
> We drill to know we're not alone
> Good friends, good books, and a drill cluster: this is the ideal life
> Outside of a dog, Drill is man's best friend
>
>
> --
> Sender:Arina Yelchiyeva 
> Sent at:2018 Sep 11 (Tue) 10:27
> To:user 
> Cc:dev 
> Subject:Re: [IDEAS] Drill start up quotes
>
> Some quotes ideas:
>
> drill never goes out of style
> everything is easier with drill
>
> Kunal,
> regarding config, sounds reasonable, I'll do that.
>
> Kind regards,
> Arina
>
>
> On Tue, Sep 11, 2018 at 12:17 AM Benedikt Koehler 
> wrote:
>
> > You told me to drill sergeant! (Forrest Gump)
> >
> > Benedikt
> > @furukama
> >
> >
> > Kunal Khatua  schrieb am Mo. 10. Sep. 2018 um 21:01:
> >
> > > +1 on the suggestion.
> > >
> > > I would also suggest that we change the backend implementation of the
> > > quotes to refer to a properties file (within the classpath) rather than
> > > have it hard coded within the SqlLine package.  This will ensure that
> new
> > > quotes can be added with every release without the need to touch the
> > > SqlLine fork for Drill.
> > >
> > > ~ Kunal
> > > On 9/10/2018 7:06:59 AM, Arina Ielchiieva  wrote:
> > > Hi all,
> > >
> > > we are close to SqlLine 1.5.0 upgrade which now has the mechanism to
> > > preserve Drill customizations. This one does include multiline support
> > but
> > > the next release might.
> > > You all know that one of the Drill customizations is quotes at
> startup. I
> > > was thinking we might want to fresh up the list a little bit.
> > >
> > > Here is the current list:
> > >
> > > start your sql engine
> > > this isn't your grandfather's sql
> > > a little sql for your nosql
> > > json ain't no thang
> > > drill baby drill
> > > just drill it
> > > say hello to my little drill
> > > what ever the mind of man can conceive and believe, drill can query
> > > the only truly happy people are children, the creative minority and
> drill
> > > users
> > > a drill is a terrible thing to waste
> > > got drill?
> > > a drill in the hand is better than two in the bush
> > >
> > > If anybody has new serious / funny / philosophical / creative quotes
> > > ideas, please share and we can consider adding them to the existing
> list.
> > >
> > > Kind regards,
> > > Arina
> > >
> > --
> >
> > --
> > Dr. Benedikt Köhler
> > Kreuzweg 4 • 82131 Stockdorf
> > Mobil: +49 170 333 0161 • Telefon: +49 89 857 45 84
> > Mail: bened...@eigenarbeit.org
> >
>

[IDEAS] Drill start up quotes

[Arina Ielchiieva]

Hi all,

we are close to SqlLine 1.5.0 upgrade which now has the mechanism to
preserve Drill customizations. This one does include multiline support but
the next release might.
You all know that one of the Drill customizations is quotes at startup. I
was thinking we might want to fresh up the list a little bit.

Here is the current list:

start your sql engine
this isn't your grandfather's sql
a little sql for your nosql
json ain't no thang
drill baby drill
just drill it
say hello to my little drill
what ever the mind of man can conceive and believe, drill can query
the only truly happy people are children, the creative minority and drill
users
a drill is a terrible thing to waste
got drill?
a drill in the hand is better than two in the bush

If anybody has new serious / funny /  philosophical / creative quotes
ideas, please share and we can consider adding them to the existing list.

Kind regards,
Arina

[ANNOUNCE] New PMC member: Charles Givre

[Arina Ielchiieva]

I am pleased to announce that Drill PMC invited Charles Givre to the PMC
and he has accepted the invitation.

Congratulations Charles and welcome to PMC squad :)

- Arina
(on behalf of Drill PMC)

[ANNOUNCE] New Committer: Weijie Tong

[Arina Ielchiieva]

The Project Management Committee (PMC) for Apache Drill has invited Weijie
Tong to become a committer, and we are pleased to announce that he has
accepted.

Weijie Tong has become a very active contributor to Drill in recent months.
He contributed the Join predicate push down feature which will be available
in Apache Drill 1.15. The feature is non trivial and has covered changes
to all aspects of Drill: RPC layer, Planning, and Execution.

Welcome Weijie, and thank you for your contributions!

- Arina
(on behalf of Drill PMC)

Re: [DISCUSS] Deprecation policy in Drill

[Arina Ielchiieva]

Hi all,

thanks for your feedback. Looks like the majority here is considering Drill
options and tables as part of API.
Based on that when we deprecate any of such notions we should not remove
them to ease updates and not break backward compatibility.
Though when Drill will move to 2.0 (which is next major version, not ETA
yet) where API might be changed, deprecated notions should be considered to
be removed as well.
Does this sound reasonable?

Kind regards,
Arina

On Mon, Aug 27, 2018 at 7:42 PM salim achouche  wrote:

> Drill is a SQL engine, which means the SQL syntax and associated options
> (runtime configuration and session properties) constitute its user facing
> APIs (if I may say). When we talk about deprecating and then removing
> documented session / configuration properties within the same release, then
> what does the versioning scheme mean? are we suggesting that:
> - Options are not part of the main Drill APIs?
> - It is ok to deprecate and then rename such properties (not remove)?
> - It is ok to deprecate and then remove existing properties as long as
> there is no change in behavior?
>

[ANNOUNCE] New PMC member: Boaz Ben-Zvi

[Arina Ielchiieva]

I am pleased to announce that Drill PMC invited Boaz Ben-Zvi to the PMC and
he has accepted the invitation.

Congratulations Boaz and thanks for your contributions!

- Arina
(on behalf of Drill PMC)

[DISCUSSION] current project state

[Arina Ielchiieva]

Hi all,

as a new PMC Chair I would like to thank users for choosing and using
Apache Drill and contributors /  committers for making improvements and
fixes. Recently Apache Drill 1.14 was released bundled up with many
improvements and new features. Please feel free to try it out and share
your experience. As always we would love to hear your success stories of
using Apache Drill.

Also I encourage users to share any problems found in Drill, as well as any
suggestions for future improvements. Feel free to start discussion on the
mailing list and then file a Jira with the summary. Contributions are
always welcome: minor, major, doc improvements or grammar fixes. Just file
a Jira and open the PR. Do not hesitate to ping developers on the mailing
list if PR is not being timely reviewed.

Latest project reports show:
Apache Drill project has healthy release schedule, each release includes
lots of features.
Mailing list (user / dev) are getting substantial support from the active
developers, including Stackoverflow and Twitter.
New committers are added on the steady basis.

Overall project is growing and moving forward. There have been discussions
about Drill 2.0 last year and currently Drill metastore feature is under
active investigation which might the breaking change for 2.0.

Please feel free to reply to this email with your comments / concerns /
ideas about current project state.

Kind regards,
Arina

Drill Hangout tomorrow at 10 am PST

[Arina Ielchiieva]

 We will have our bi-weekly hangout tomorrow July 10th at 10 am PST.
Please reply to this post with proposed topics to discuss.

Hangout link:
https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

Kind regards,
Arina

Re: [ANNOUNCE] Apache Drill release 1.13.0

[Arina Ielchiieva]

One more point, from this release Apache Drill no longer supports JDK7 and
fully moved to JDK8.

Kind regards
Arina

On Mon, Mar 19, 2018 at 5:51 AM, Abhishek Girish  wrote:

> Congratulations everyone, on yet another great release of Apache Drill!
> On Mon, Mar 19, 2018 at 6:57 AM Parth Chandra  wrote:
>
> > On behalf of the Apache Drill community, I am happy to announce the
> > release of
> > Apache Drill 1.13.0.
> >
> > For information about Apache Drill, and to get involved, visit the
> > project website
> > [1].
> >
> > This release of Drill provides the following new features and
> improvements:
> >
> > - YARN support for Drill [DRILL-1170
> > ]
> >
> > - Support HTTP Kerberos auth using SPNEGO [DRILL-5425
> > ]
> >
> > - Support SQL syntax highlighting of queries [DRILL-5868
> > ]
> >
> > - Drill should support user/distribution specific configuration checks
> > during startup [DRILL-6068
> > ]
> >
> > - Upgrade DRILL to Calcite 1.15.0 [DRILL-5966
> > ]
> >
> > - Batch Sizing improvements to reduce memory footprint of operators
> >
> > - [DRILL-6071 <
> > https://issues.apache.org/jira/browse/DRILL-6071>]
> > - Limit batch size for flatten operator
> >
> > - [DRILL-6126 <
> > https://issues.apache.org/jira/browse/DRILL-6126>]
> > - Allocate memory for value vectors upfront in flatten operator
> >
> > - [DRILL-6123 <
> > https://issues.apache.org/jira/browse/DRILL-6123>]
> > - Limit batch size for Merge Join based on memory.
> >
> > - [DRILL-6177 <
> > https://issues.apache.org/jira/browse/DRILL-6177>]
> > - Merge Join - Allocate memory for outgoing value vectors based on sizes
> of
> > incoming batches.
> >
> >
> > For the full list please see release notes [2].
> >
> > The binary and source artifacts are available here [3].
> >
> > Thanks to everyone in the community who contributed to this release!
> >
> > 1. https://drill.apache.org/
> > 2. https://drill.apache.org/docs/apache-drill-1-13-0-release-notes/
> > 3. https://drill.apache.org/download/
> >
>

Re: Working on upgrade Drill Calcite version

[Arina Ielchiieva]

I am happy to announce that Apache Drill has rebased on Calcite 1.15. All
information can be found in DRILL-3993 and in Google Doc [1].
Currently changes are in master and are intended to be part of Drill 1.13.0
release unless something critical pops up. Feel free to test them and give
your feedback.

Thanks Roman Kulyk and Volodymyr Vysotskyi for making this happen.
Aman Sinha and Chunhui Shi for the code review.
And of course Julian Hyde for helping to address / review Calcite specific
issues.

[1]
https://docs.google.com/document/d/1Lqk9NoKQviz0YimBmov4z1pui7QjJGjDVwMa1p0emPk/edit#heading=h.i3rowg20vxv4


Kind regards
Arina

On Fri, Sep 1, 2017 at 8:01 PM, Roman Kulyk  wrote:

> Team, thanks for your answers!
>
> Julian, сontinuing the topic of Calcite tests I just want to note that I
> have ignored 7 test in 5ba68240 commit in Calcite. Here is a list of
> ignored tests:
>
>   TraitConversionTest.testTraitConversion
>   JdbcAdapterTest.testCrossJoinWithJoinKeyPlan
>   JdbcAdapterTest.testCrossJoinWithJoinKeyAndFilterPlan
>   JdbcTest.testJoinManyWay (it was also ignored by 8a87ad1e5)
>   LatticeTest.testTileAlgorithm (it was also ignored by 2e076e1a)
>   LatticeTest.testTileAlgorithm2 (it was also ignored by 8a87ad1e5)
>   LatticeTest.testTwoLattices
>
> You can see detailed information in my comments. Also, I want to notice
> that there are 2 required commits in my Calcite branch: 5ba68240 and
> 1f5145b1. Without them, we lose a big part of Drill functionality.
>
> Best regards,
> Roman Kulyk
>
> 2017-08-31 18:43 GMT+00:00 Julian Hyde :
>
> > +dev@calcite.
> >
> > Excellent work, Roman.
> >
> > Your changes to Calcite don’t seem to be very major, and I ran the test
> > suite on your https://github.com/KulykRoman/incubator-calcite/commits/
> > DrillCalcite1.13.0_rc1 branch and all the tests pass. So when you are
> > ready, I think we can expedite getting these changes back into Calcite.
> >
> > Calcite will release 1.14 probably in the next two weeks. So, when you’re
> > on to 1.13 let’s continue the momentum and get up to the latest release.
> I
> > would love to get to the situation that we can ask Drill to verify a
> > Calcite release before we make it. There are possibly breaking changes in
> > the area of schema and grouping-sets aggregate but it should be mostly
> > straightforward.
> >
> > Julian
> >
> >
> > > On Aug 31, 2017, at 8:12 AM, Jinfeng Ni  wrote:
> > >
> > > It's great to hear that someone is working on moving Drill on new
> Calcite
> > > release! As someone who did similar job in last two rebase work, I
> could
> > > see that there would be many regressions to fix. Please let me know if
> I
> > > can provide any help.
> > >
> > > I thought we have pushed almost Drill specific commits to Calcite. I'll
> > > take a look at your Calcite branch. Ideally, we want to get rid of all
> > > Drill specific commits, after the rebase work this time.
> > >
> > > Thanks,
> > >
> > > Jinfeng
> > >
> > >
> > >
> > >
> > > On Thu, Aug 31, 2017 at 8:04 AM, Aman Sinha 
> > wrote:
> > >
> > >> Yes, this is long overdue !  Thanks for working on it Roman.   If
> > needed,
> > >> we can do a separate hangout with a few selected folks who have worked
> > on
> > >> Drill+Calcite to provide feedback on how to resolve the test failures.
> > >> Although, at this stage you probably know a lot more about the
> > integration
> > >> than many of us.
> > >>
> > >> -Aman
> > >>
> > >> On Thu, Aug 31, 2017 at 7:33 AM, Roman Kulyk 
> > wrote:
> > >>
> > >>> Hi, team!
> > >>>
> > >>> I want to inform that I am working on updating Drill Calcite version
> to
> > >>> 1.13. I forked from Calcite "branch-1.13" and added some commits:
> > >>> - DRILL-1455: Add return type-inference strategy for arithmetic
> > operators
> > >>> when one of the arguments is ANY type. (was not in Calcite)
> > >>> - Add new method to ViewExpander interface to allow passing
> SchemaRoot.
> > >>> (was not in Calcite)
> > >>> - Allow a MAP literal type. (was not in Calcite)
> > >>> - DRILL-4047: Modify table functions to allow querying a table with
> > >> options
> > >>> in Drill (was not in Calcite)
> > >>> - Drill-specific change: Add back AbstractConverter in RelSet.java
> ...
> > >>> (small changes after a3bc0d8 commit)
> > >>> - CALCITE-628 related but not fix the problem: Ensure target traits
> are
> > >>> simple when use Frameworks or RelOptRule.convert() method. (small
> > changes
> > >>> after a3bc0d8 commit)
> > >>> -  Support select * from schema-less table in execution engine like
> > Drill
> > >>> (small changes after 9bd7d75 commit)
> > >>>
> > >>> Only after adding these commits to Calcite, a part of  Drill
> > >> functionality
> > >>> started to work as before. Currently, I am at Drill integration
> stage.
> > I
> > >> am
> > >>> fixing Drill unit tests. Progress can be tracked in my branches.
> > >>>
> > >>> Drill: 

[SECURITY] CVE-2017-12630 Apache Drill XSS vulnerability

[Arina Ielchiieva]

*CVE-2017-12630 Apache Drill XSS vulnerability*

*Severity*: Important

*Vendor:* The Apache Software Foundation

*Versions Affected:*
Apache Drill 1.11.0 and earlier

*Description*
In Apache Drill 1.11.0 and earlier when submitting form from Query page
users are able to pass arbitrary script or HTML which will take effect on
Profile page afterwards.

Example:
After submitting special script that returns cookie information from Query
page, malicious user may obtain this information from Profile page
afterwards.

*Mitigation:*
Users of the affected versions should upgrade to Apache Drill to 1.12.0 and
later.

*Credit:*
Sanjog Panda

Kind regards
Arina

[ANNOUNCE] Apache Drill 1.12.0 Released

[Arina Ielchiieva]

On behalf of the Apache Drill community, I am happy to announce the release
of Apache Drill 1.12.0.

For information about Apache Drill, and to get involved, visit the project
website [1].

This release of Drill provides the following new features and improvements:

Kafka and OpenTSDB Storage Plugins (DRILL-4779, DRILL-5337)
Queue-Based Memory Assignment for Buffering Operators (Throttling)
(DRILL-5716)
Networking Functions (DRILL-5834)
SSL Support (DRILL-5431)
Network Encryption Support (DRILL-5682)
Access to Paths Outside the Current Workspace (DRILL-5964)

For the full list please see release notes [2].

The binary and source artifacts are available here [3].

Thanks to everyone in the community who contributed to this release!

1. https://drill.apache.org/
2. https://drill.apache.org/docs/apache-drill-1-12-0-release-notes/
3. https://drill.apache.org/download/