Apache Drill Hangout - Sept 17, 2019

Hi Drillers, We will have our bi-weekly hangout tomorrow Sept 17th, at 10 AM PST (link: https://meet.google.com/yki-iqdf-tai ). If there are any topics you would like to discuss during the hangout please respond to this email. Thanks, Gautam

Re: [ANNOUNCE] New PMC Chair of Apache Drill

Thank you Arina! Congratulations Charles on becoming the new PMC Chair! Gautam On Thu, Aug 22, 2019 at 12:43 PM Ellen Friedman wrote: > Thank you Arina! > > Charles, congratulations on your new role with Drill! > > Cheers, > Ellen > > On Thu, Aug 22, 2019 at 10:30 AM Charles Givre wrote: > >

Apache Drill Hangout - Aug 20, 2019

Hi Drillers, We will have our bi-weekly hangout tomorrow Aug 20th, at 10 AM PST (link: https://meet.google.com/yki-iqdf-tai ). If there are any topics you would like to discuss during the hangout please respond to this email. Thanks, Gautam

Re: [ANNOUNCE] New Committer: Bohdan Kazydub

Congratulations Bohdan! Gautam On Mon, Jul 15, 2019 at 11:53 PM Bohdan Kazydub wrote: > Thank you all for your support! > > On Tue, Jul 16, 2019 at 4:16 AM weijie tong > wrote: > > > Congrats Bohdan! > > > > On Tue, Jul 16, 2019 at 12:54 AM Vitalii Diravka > > wrote: > > > > > Congrats

Re: [ANNOUNCE] New PMC member: Sorabh Hamirwasia

Congratulations Sorabh! Gautam On Fri, Apr 5, 2019 at 9:06 AM Arina Ielchiieva wrote: > I am pleased to announce that Drill PMC invited Sorabh Hamirwasia to > the PMC and > he has accepted the invitation. > > Congratulations Sorabh and welcome! > > - Arina > (on behalf of Drill PMC) >

Apache Drill Hangout - March 19, 2019

Hello All, Does anyone have any topics to discuss during the hangout tomorrow? We will start the hangout at 10AM PST (http://meet.google.com/yki-iqdf-tai). Thanks, Gautam

Re: Anouncing Powered-By-Drill page

Great work team! Looks great :) Gautam On Thu, Feb 28, 2019 at 11:56 AM Jyothsna Reddy wrote: > This is some awesome addition to the website!! > > > > On Thu, Feb 28, 2019 at 9:38 AM Pritesh Maker wrote: > > > This is awesome! Thanks for driving this and also to Aashreya for his > >

Re: Problem Creating Views In Apache Drill

Hi Peter, Do you see the same error when you run the query instead? SELECT id, settlement, settlement_type, to_timestamp(to_char(settlement_date, '#')) as settlement_date ,transaction_id, transaction_type, settlement_institution, settlement_institution_bank_domain_id, domain_i,product_group_id,

Re: [ANNOUNCE] New Committer: Karthikeyan Manivannan

Congratulations Kathik! Well-deserved :) Gautam On Fri, Dec 7, 2018 at 11:12 AM Abhishek Girish wrote: > Congratulations Karthik! > > On Fri, Dec 7, 2018 at 11:11 AM Arina Ielchiieva wrote: > > > The Project Management Committee (PMC) for Apache Drill has invited > > Karthikeyan > >

Re: Write custom aggregate function

Hi Andrea, Hi, > > I tried to implement it using ObjectHolder and ComplexWriter and it doesn't > seem possible to achieve something general at this point. > > Andrea I would think it should be possible to achieve what you are trying to do with the ObjectHolder and ComplexWriter. Could you

Re: [ANNOUNCE] New Committer: Hanumath Rao Maduri

Congratulations Hanumath! Well deserved :) Gautam On Thu, Nov 1, 2018 at 9:44 AM AnilKumar B wrote: > Congratulations Hanumath. > > Thanks & Regards, > B Anil Kumar. > > > On Thu, Nov 1, 2018 at 9:39 AM Vitalii Diravka wrote: > > > Congratulations! > > > > Kind regards > > Vitalii > > > > > >

Re: How to resolve error- "One or more nodes ran out of memory while executing the query"?

Hi Ashish, Can you please share the query profile? I am curious - did you start with hash_join disabled or did it for troubleshooting? Gautam On Thu, Oct 25, 2018 at 12:01 AM Ashish Pancholi wrote: > -- Forwarded message - > From: Ashish Pancholi > Date: Thu, Oct 25, 2018 at

Re: Own storage plugin

Hi Alex, Thank you for using Drill and also planning to contribute to it. Here are a few resources [1], [2], [3] which would be helpful to get you started. Also it would be useful to look at one/two existing plugins to familiarize yourself with the code. [1]

Re: Question on querying partitioned data

What you are looking for is run-time partition pruning instead of compile-time partition pruning. AFAIK Drill currently supports compile-time partition pruning. On Thu, Jul 12, 2018 at 5:06 PM, Gauri Srinivas wrote: > Hi, > > I'm trying to use drill on data partitioned by day. As I understand

Re: [HANGOUT] Topics for May 15 2018

Hangout link: https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_hangouts_-5F_event_ci4rdiju8bv04a64efj5fedd0lc=DwIFAw=cskdkSMqhcnjZxdQVpwTXg=jGaWXfAULy7L7yLSDM6rFQ=WED3lepY_zTnY-lDfHxhDWIlMw3VKANkW0iQu5FFrkI=N0UtWs-KDoE5koI9hSiULag28iNA5JJfXOLB5JnNv-0=

[HANGOUT] Topics for 05/01/2018

Hi all, We will have a Drill hangout tomorrow (Tuesday May 1) at 10 AM Pacific Time. Please suggest topics by replying to this thread or bring them up during the hangout. Hangout link: https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Thanks, Gautam

Re: Accessing underlying scheme of input

I am not sure I understood correctly but if you want input types you can use the typeof function. e.g. SELECT typeof(employee_id) FROM cp.`employee.json` LIMIT 3; +-+ | EXPR$0 | +-+ | BIGINT | | BIGINT | | BIGINT | +-+ Gautam

Re: Question about how Drill optimizes the queries and splits the loads in HDFS cluster?

Hi LittleCho, These notes might be helpful: https://drill.apache.org/docs/architecture-introduction/ https://drill.apache.org/docs/drill-query-execution/ Also, please look into the notes for

Re: Consuming Returned Results

Hi Loubna, I tried a sample query which returns approximately 10,000 records and it works for me. I am not familiar with your use case but maybe using LIMIT/OFFSET to achieve pagination might help instead of requesting the entire data. I am assuming your data would not be constrained to 100K

[HANGOUT] Topics for 10/31/2017

Hi, We will have a Drill hangout tomorrow (Tuesday Oct 31) at 10 AM Pacific Time. Please suggest topics by replying to this thread or bring them up during the hangout. Hangout link: https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Thanks, Gautam

Re: Question about Drill aggregate queries and schema change

Hi Cliff, Thanks so much for trying it out. The error means that this particular operator assumed the datatype as `nullable int` since the first batch it saw did not have data for some column (`a` in you case?) However, in one of the subsequent batches it sees the datatype is a `big int`.

Re: [HANGOUT] Topics for 05/30/17

From: Gautam Parai <gpa...@mapr.com> Sent: Monday, May 29, 2017 12:59:37 PM To: d...@drill.apache.org; user@drill.apache.org Subject: [HANGOUT] Topics for 05/30/17 Hi Drillers, The bi-weekly Drill hangout is scheduled for tomorrow (05/30/17 10 AM PT). Please suggest

[HANGOUT] Topics for 05/30/17

Hi Drillers, The bi-weekly Drill hangout is scheduled for tomorrow (05/30/17 10 AM PT). Please suggest topics by adding them to this thread. We will also ask for topics at the beginning of the hangout. Thanks, Gautam

Re: Writing to s3 using Drill

Hi Shuporno, Did you try following the suggestions from Abhishek? Please let us know your observations. Also, please share the CTAS command you are using to write to s3. Gautam From: Shuporno Choudhury Sent: Thursday, May

Re: Does s3 plugin support AWS S3 signature version 4 ?

Another Drill user recently filed this bug for a similar issue: (DRILL-5536) Support AWS S3 SSE-KMS encrypted objects querying Gautam From: Abhishek Girish Sent: Wednesday, May 24, 2017 12:38:14 PM To: user@drill.apache.org Subject: Re:

Re: S3 configuration for ceph or atmos

Hi Raz, Please see here for an example https://drill.apache.org/docs/s3-storage-plugin/ Gautam From: yousef.l...@gmail.com on behalf of Raz Baluchi Sent: Wednesday, May 24, 2017 7:03:12 AM To:

Re: querying aws s3 objects that are server side encrypted

Hi Chandrakanth, A quick google got me this result from AWS: "We are sorry you have trouble getting this to work. The error is consistent with KMS requiring SigV4. We are still looking at why, by default, the s3 client for node.js doesn't do that. In the interim, you can explicitly set the

Re: Writing to s3 using Drill

Hi Shuporno, Could you please specify the configuration from S3 storage plugin here and the output of `show schemas` as it pertains to s3? Is `writable` set to true? Gautam From: Shuporno Choudhury Sent: Monday, May 22, 2017

Re: Running cartesian joins on Drill

Can you please specify the query you are trying to execute? Gautam From: Muhammad Gelbana Sent: Saturday, May 6, 2017 7:34:53 AM To: user@drill.apache.org; d...@drill.apache.org Subject: Running cartesian joins on Drill Is there a reason

Re: Minimise query plan time for dfs plugin for local file system on tsv file

Can you please provide the drillbit.log file? Gautam From: PROJJWAL SAHA Sent: Monday, March 6, 2017 1:45:38 AM To: user@drill.apache.org Subject: Fwd: Minimise query plan time for dfs plugin for local file system on tsv file all, please

Re: Drill Hangout 2/7/2017

Hi Jasbir, Which version of Drill are you using? This issue was fixed in 1.8.0. Please see DRILL-3710 As Jinfeng mentioned, the default value of `planner.in_subquery_threshold` is 20. So you should see the Join plan for ~300 in clause size.

Re: UDF parameters

Yes, the ObjectHolder can store any arbitrary object. You are responsible for casting it into the appropriate type when reading from the ObjectHolder. But, I think it is listed as deprecated. Gautam From: Muhammad Gelbana Sent: Monday,

Re: MySQL CONNECTION_ID() equivalent in Drill

You could consider contributing a UDF (user-defined function) for the same. If the UserSession were to contain a unique ID per session it is pretty straightforward to pass it along to the UDF which would return it to the caller. After this, you could then invoke it just like MySQL e.g. *SELECT

[HANGOUT] Suggestions for topics for 11/15

Hi all, The bi-weekly hangout is tomorrow (11/15/16, 10 AM PST). If you have any suggestions for topics for tomorrow, please add to this thread. We will also ask for topics at the beginning of the hangout. Hangout link: https://hangouts.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

Re: IN operator can take how many inputs ?

Yes, this was fixed in Drill 1.8 - please let us know if this does not work in 1.8. As Jinfeng mentioned earlier, the option can be set to a very large value (LONG type). Gautam On Fri, Oct 7, 2016 at 2:29 AM, Tushar Pathare wrote: > Hello, > > I think it is fixed in 1.8

Re: UDF IndexOutOfBoundsException for large String

Can you try the following @Param FieldReader in; int type = in.getType().getMinorType().getNumber(); *if (*type == org.apache.drill.common.types.TypeProtos.MinorType. VARCHAR_VALUE) { String res = in.readText().toString(); } and see if it resolves your issue? On Mon, Sep 26, 2016 at

Re: Cannot cancel query - stuck in CANCELLATION_REQUESTED

Can you please run EXPLAIN PLAN FOR your_query? The plan will give an idea if there is something unusual and it will also confirm if this is the same issue as DRILL-4355 (if your query hangs during planning). Gautam On Mon, Sep 26, 2016 at 3:29 AM, Khurram Faraaz wrote:

Re: Question regarding drill Query

It might be helpful to split these IDs into ranges. Is it possible to do so? On Thu, Sep 22, 2016 at 12:02 PM, Ranjit Shukla wrote: > Hi > > Question: > > *How can I configure drill to accept large query length?* > I am facing this difficulty while querying with

Re: Suggestions for hangout topics for 08/09

mal implementation is vulnerable to overflow errors - To address issues and simplify implementation Aman - Good idea to look at previous design TODO: Parth/Aman Attach - Mehant's doc to JIRA Please correct me if I missed anything. Gautam On Tue, Aug 9, 2016 at 9:59 AM, Gautam Parai <gpa...@maprte

Re: Suggestions for hangout topics for 08/09

estions about Subbu's pull request? > > > > -- Zelaine > > > > > > On Mon, Aug 8, 2016 at 11:33 AM, Gautam Parai <gpa...@maprtech.com> > wrote: > > > >> Tomorrow's hangout is scheduled for 10AM - 11AM PST > >> > >> On Mon, Aug 8, 20

Re: Suggestions for hangout topics for 08/09

Tomorrow's hangout is scheduled for 10AM - 11AM PST On Mon, Aug 8, 2016 at 11:30 AM, Subbu Srinivasan <ssriniva...@zscaler.com> wrote: > What time is tomorrow's mtg scheduled for? > > > On Mon, Aug 8, 2016 at 10:48 AM, Gautam Parai <gpa...@maprtech.com> wrote: > >

Suggestions for hangout topics for 08/09

If you have any suggestions for Drill hangout topics for tomorrow, you can add it to this thread. We will also ask around at the beginning of the hangout for any topics. We will try to cover whatever possible during the 1 hr. Topics: 1. DRILL-4653: Malformed JSON should not stop the entire

Re: Drill 1.6 Cartesian or an inequality joins are not supporting ?

Do you see the following error "SYSTEM ERROR: This query cannot be planned possibly due to either a cartesian join or an inequality join"? If so, then there is an existing Jira (DRILL-2915) for the issue. On Fri, Aug 5, 2016 at 10:12 AM, Jinfeng Ni wrote: > AFAIK, Drill

Re: [ANNOUNCE] New PMC Chair of Apache Drill

Congratulations Parth! On Wed, May 25, 2016 at 9:02 AM, Jinfeng Ni wrote: > Big congratulations, Parth! > > Thank you, Jacques, for your contribution and leadership over the last > few years! > > > On Wed, May 25, 2016 at 8:35 AM, Jacques Nadeau >