Hangout Topics for 04/30/2019

Hello Everyone, Does anyone have any topics for today's hangout? Sorry for sending late notification, topics will be accepted on call as well. Hangout will start at 10 AM PST and you can use this link to join: http://meet.google.com/yki-iqdf-tai) Thanks, Sorabh

Re: Apache Drill Hangout - March 19, 2019

Hi Gautam, 2 topics which we can discuss are: - 2.0 Branch - 1.16 release cut-off date based on tasks in progress. Thanks, Sorabh On Mon, Mar 18, 2019 at 11:31 AM Gautam Parai wrote: > Hello All, > > Does anyone have any topics to discuss during the hangout tomorrow? > We will start the

Re: Performace issue

Data is not stored in the disk. All data > > is in the memory. > > > > Thanks, > > Praveen > > > > On Tue, Feb 12, 2019 at 11:49 PM Sorabh Hamirwasia < > > sohami.apa...@gmail.com> wrote: > > > >> Hi Praveen, > >> Can you also s

Re: Performace issue

Hi Praveen, Can you also share what is the schema of your entire dataset and in what format it's stored? Thanks, Sorabh On Tue, Feb 12, 2019 at 10:02 AM Kunal Khatua wrote: > You'll need to edit the memory settings in DRILL_HOME/conf/drill-env.sh > I suspect that your 5MB JSON data might be

Drill Resource Management Project

Hi All, Please find below the link[1] to the slides for Drill Resource Management project which Hanu, Karthik and I presented on this week's hangout. For details you can refer to design document at [2]. The implementation details section of design document is still WIP. All the work for this

Re: Apache Drill & LDAP

Hi Alaa, Drill provides support for using PAM modules for authentication. By default it only uses unix user for the authentication and enforces authentication rules as defined in sudo and login pam profiles[1]. But PAM can also be configured to use ldap using the provided module libpam-ldap for

Re: November Apache Drill board report

Hi Arina, Lateral/Unnest feature was part of 1.14 though it was disabled by default. Should we mention it as part of 1.14 enhancements in the report? Thanks, Sorabh On Thu, Nov 1, 2018 at 9:29 AM Arina Yelchiyeva wrote: > Thanks, Aman! Updated the report. > I went too far with 2019, luckily

Re: Using Drill 1.14 with a kerberized cluster

Hi, For both Drill RPC and Http path, authentication is enabled using a single a parameter *security.user.auth.enabled:true,* which is true in your setup. You have correctly configured auth mechanism as KERBEROS for Drill RPC path but for HTTP path the corresponding mechanism will be SPNEGO and

Re: [ANNOUNCE] New Committer: Hanumath Rao Maduri

Congratulations Hanu! Thanks, Sorabh On Thu, Nov 1, 2018 at 1:35 PM Hanumath Rao Maduri wrote: > Thank you all for the wishes! > > Thanks, > -Hanu > > On Thu, Nov 1, 2018 at 1:28 PM Chunhui Shi .invalid> > wrote: > > > Congratulations Hanu! > >

Re: Query hung in Cancelled_Requested state

Hi Divya, If possible can you also try running same query on master ? There have been lots of improvement/bug fixes in cancellation code path to resolve hang situations post 1.10. Thanks, Sorabh On Thu, Oct 25, 2018 at 10:49 PM Kunal Khatua wrote: > What is the Drillbit's JStack? You can try

Re: Using Drill embedded as a library

Hi David, Can you try using drill-jdbc as dependency instead of jdbc-all ? Reason being jdbc-all only contains client side dependencies and also has jdbc driver class in it. All other dependencies inside jdbc-all are shaded to have oadd prefix. When you load Driver class it's getting loaded from

Re: Windows embedded mode - query

Hi Govind, Enabling security from Drill's perspective should be same for all the platforms. For example: If you are using PAM then you can follow the same process to configure Drill listed here [1]. But as pam verifies DrillClient user with os user you need to make sure that os user is present on

Re: [HANGOUT] Topics for 10/16/2018

- There were list of topics which were identified to present in future hangout sessions. With each hangout invitation we will announce the topics which are finalized for that week. Thanks, Sorabh On Mon, Oct 15, 2018 at 11:05 AM Sorabh Hamirwasia wrote: > Resending with correct date (10

Re: is there some place to question or discuss about apache drill?

Hi, You can use Drill user or dev mailing list for discussion based on the topics of your interest. We also host Drill Hangout session bi-weekly to discuss any topics. There is one tomorrow (10/16) and would be great if you can join using link at [1]. [1]: https://meet.google.com/yki-iqdf-tai

Re: [HANGOUT] Topics for 10/16/2018

Resending with correct date (10/16) in subject. On Mon, Oct 15, 2018 at 10:59 AM Sorabh Hamirwasia wrote: > Hi All, > > We will have Apache Drill Hangout tomorrow (October 16, 2018) at 10 AM > PST. Please suggest any topic you will be interested in discussing by > replying to

[HANGOUT] Topics for 10/15/2018

Hi All, We will have Apache Drill Hangout tomorrow (October 16, 2018) at 10 AM PST. Please suggest any topic you will be interested in discussing by replying to this thread. We will also accept additional topics at the beginning of the hangout. Please use the below link to join:

Re: [ANNOUNCE] New Committer: Chunhui Shi

Congratulations Chunhui!! Thanks, Sorabh On Fri, Sep 28, 2018 at 12:56 PM Paul Rogers wrote: > Congrats Chunhui! > > Thanks, > - Paul > > > > On Friday, September 28, 2018, 2:17:42 AM PDT, Arina Ielchiieva < > ar...@apache.org> wrote: > > The Project Management Committee (PMC) for Apache

Re: unable to login to Drill as admin user all other login works fine

Can you help to see if there is any logs or info in drillbit.log/drillbit.out. ? Also which version of Drill are you running ? I am assuming that this admin user has same UID and password on all the Drillbit nodes in cluster. Thanks, Sorabh On Wed, Jun 20, 2018 at 6:21 PM, Divya Gehlot wrote:

Re: A problem about "Using libpam4j as the PAM Authenticator ."

Hi, Since you have enabled PLAIN authentication in configuration using libpam4j you have to pass username and password from client to authenticate. The way embedded mode in Drill works is running an embedded Drillbit within sqlline client. Hence to connect using embedded mode you have to pass

Re: strange access control exception

Hi Divya, Can you please share the stack trace from drillbit.out/drillbit.log file for AccessControlException ? And also the query profile will be helpful. Thanks, Sorabh From: Divya Gehlot Sent: Monday, June 18, 2018 3:48:43 AM To: user@drill.apache.org

Re: Unable to run Drill queries on Drill1.13

Hi Peter, Since you mentioned seeing a username textbox I think you are using WebUI to run query. Your setup must be unsecure with impersonation enabled. You can disable impersonation on Drillbit side by removing all the settings under drill.exec.impersonation block from drill-override.conf

Re: OS user and AD user works simultaneously in Drill

Hi Divya, You can create a custom PAM profile to allow that and place that in /etc/pam.d/ location. Configure Drill to use that pam-profile by using below parameter in drill-override.conf. drill.exec.security.user.auth.pam_profiles: ["sudo", "login", "customProfile"] Basically in your PAM

Re: How to Start Drill Service From Linux Non-root user

to write only few records even then same error is repeating. Please help. Best regards, _ Tilak Surneni -Original Message- From: Sorabh Hamirwasia [mailto:shamirwa...@mapr.com] Sent: Wednesday, May 30, 2018 12:32 AM To: user@drill.apache.org Subject: Re: How

Re: How to Start Drill Service From Linux Non-root user

Hi Surneni, With respect to using the service specific to a particular user other than root, do you mean executing queries as a query user rather than Drill process user ? If yes then you need to enable impersonation for that on Drillbit side [1]. Later when you connect using client tool (like

Re: tableau drill connection - handshake time out error

to connect. Add HandshakeTimeout= in the AdvancedProperties field. If you are using the Apache Drill connector in Tableau: Then you would need to create a custom settings file for Tableau to pick up this information. Thanks, Sorabh From: Sorabh Hamirwasia

Re: tableau drill connection - handshake time out error

tion file to know more about the setup. Thanks, Sorabh From: Divya Gehlot <divya.htco...@gmail.com> Sent: Tuesday, May 8, 2018 8:29:26 PM To: Sorabh Hamirwasia Cc: user@drill.apache.org Subject: Re: tableau drill connection - han

Re: permission denied error in drillbit.out

This just means it failed to access the directory /Data/Users/AP/ using that MapR file client instance. It can happen if the MapR file client instance is created in context of a user who doesn't have read permission for that directory. Probably you might have some log before this line stating

Re: tableau drill connection - handshake time out error

-tracing-options?view=sql-server-2017 Thanks, Sorabh From: Divya Gehlot <divya.htco...@gmail.com> Sent: Tuesday, May 8, 2018 3:05 AM To: user@drill.apache.org; Sorabh Hamirwasia Subject: Re: tableau drill connection - handshake time out error Hi, I don't see any

Re: tableau drill connection - handshake time out error

Hi Divya, Can you please share the error seen on client side and also debug/error logs from server side (from drillbit.log and drillbit.out) ? Also what is the setup of your environment ? Are you using Tableau Server or Desktop version ? Thanks, Sorabh From:

Re: ldap, kerberos zookeeper and drill integration - Drill failing to authenticate

Hi Spiro, For the error while connecting using sqlline:- Do you have TGT generated for your client user which you are using to connect to Drill ? If yes can you check if sqlline process user has access to that ticket file or not ? Can you please share your sqlline command? Also can you share

Re: Apache Drill- Error setting up remote intermediate fragment execution

) ~[netty-common-4.0.48.Final.jar:4.0.48.Final] Thanks, Sorabh From: Sorabh Hamirwasia Sent: Wednesday, April 11, 2018 9:04 AM To: user@drill.apache.org Subject: Re: Apache Drill- Error setting up remote intermediate fragment execution Hi Surneni, This looks

Re: Apache Drill- Error setting up remote intermediate fragment execution

Hi Surneni, This looks to be an issue with your network setup. The address or hostname known to a Drillbit about another Drillbit is not resolved correctly while trying to make connection. Thanks, Sorabh From: Surneni Tilak Sent:

Re: JDBC Driver

Hi Ravi, I just want to confirm that you are using Drill's JDBC storage plugin to connect to RDS on Aws for Oracle ? You can look into [1] to see the usage. Currently JDBC plugin only supports four parameters: driver/url/username/password, there doesn't seem to be support for any separate

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

With the session option set as `drill.exec.hashagg.fallback.enabled`=TRUE; means HashAgg will not honor the operator memory limit which it was assigned to (thus not spilling to disk) and will end up consuming unbounded memory. Thanks, Sorabh From: Anup Tiwari

Re: Accessing underlying scheme of input

Hi Erol, You can run limit 0 query from client to retrieve just the schema for your input. Thanks, Sorabh From: Erol Akarsu Sent: Thursday, March 1, 2018 5:28:52 AM To: user@drill.apache.org Subject: Accessing underlying scheme of input I

Re: Unable to setup hive plugin in Drill 1.11.0

Hi Anup, >From the stack trace which you have shared it looks like you are trying to use >Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a >PR[2] open for it in 1.13 though. Also from the stack trace it's pointing towards an invalid field in HiveConf

Re: MapR Drill 1.12 Mismatch between Native and Library Versions

: a running drill bit with (it appears it's not failed > > web server, it's slow to start web server... is there a good way to > trouble > > shoot this aspect? > > > > 2018-02-09 12:33:18,575 [main] INFO o.apache.drill.exec.server.Drillbit > - > > Startup comple

Re: MapR Drill 1.12 Mismatch between Native and Library Versions

Hi John, Can you please share all your config files (*.conf and *.env) ? If I understand correctly below is your environment setting: 1) Secure MapR cluster. 2) Running drill-1.12.0 installed from MapR released RPM. 3) Re-using the configuration from drill-1.10.0. Thanks, Sorabh

Re: Apache drill validation error, table not found

Hi, Can you please try following instruction here [1] once.? Also after configuration you can try to list tables by "show databases" to make sure setup is fine. And the query should be: SELECT * FROM s3.`./data/part-r-001` LIMIT 100 [1]: https://drill.apache.org/docs/s3-storage-plugin/

Re: Fw: Re: drill in 10 minutes tutorial problem with osx

Hi Paul, The reason we are picking *org.apache* instead of *oadd.org.apache *is because using Sqlline when we search for Drilver class, it finds it first inside *jars/drill-jdbc-1.12.0.jar* instead of *jars/jdbc-driver/drill-jdbc-all-1.12.0.jar* hence it loads all the classes inside the org.apache

Re: drill in 10 minutes tutorial problem with osx

Hi Teemu, Thanks for trying out Drill. Did you downloaded the tar-ball from provided link in the tutorial ? I just tried on osx and it's working fine for me. >From the error looks like one of the class DrillConfig is missing at runtime. >Can you please check the following: 1) If the

Re: Drills' Hbase storage plugin

---+ > | true | Default schema chnaged to [hbase] | > +--+---+ > > > show tables; > (no result, took long time) <=== any idea how to solve it? > > > Regards > > > On

Re: Drills' Hbase storage plugin

Hi Ascot, I am not sure if Kerberos between Hbase storage plugin and Hbase server has been tried before or not. Link in [1] talks about Kerberos setup between Drill Client and Drillbit. However looking into the code for Hbase and link at [2], it looks like if you can put a hbase-site.xml in

Re: Drill UI (8047) Default User and Password

To add on Arina's reply: Authentication on WebServer side happens using PLAIN mechanism with PAM authenticator. So when authentication is enabled [1], you can login to WebUI using credentials of any user present on the node to which you are connecting to. For example: If you are connecting

Re: Using SASL encryption from Clients to Drillbits

Hi Knapp, SASL Plain mechanism doesn't support encryption [1] hence encryption as of now is only available via Kerberos. LDAP module which you have configured for Drill will work as an authenticator module in PLAIN mechanism and you won't be able to use encryption capabilities with it. Also

Re: Drill Session ID between Nodes

Hi John, As Paul mentioned session ID's are not global. Each session is part of the BitToUserConnection instance created for a connection between Drillbit and client. Hence it's local to that Drillbit only and the lifetime of the session is tied to lifetime of the connection. You can find the

Re: Writing to s3 using Drill

Hi Shuporno, Can you please share your S3 plugin configuration ? Looks like in your configuration you might be missing something like below: "tmp": { "location": "drill-tmp", "writable": true, "defaultInputFormat": null } Thanks, Sorabh