Re: Query hung in Cancelled_Requested state

2018-10-25 Thread Kunal Khatua
What is the Drillbit's JStack? You can try seeing the /threads webpage. On 10/25/2018 8:16:06 PM, Divya Gehlot wrote: Hi, I am using Dril 1.10 and I have query running for days and when I click on cacel the query through Drill Web UI . It hung in Cancellation_Requested state . Is there any way to

Re: [ANNOUNCE] New Committer: Hanumath Rao Maduri

2018-11-01 Thread Kunal Khatua
Congratulations, Hanu! On 11/1/2018 11:04:58 AM, Abhishek Girish wrote: Congratulations, Hanu! On Thu, Nov 1, 2018 at 10:56 AM Khurram Faraaz wrote: > Congratulations Hanu! > > On Thu, Nov 1, 2018 at 10:14 AM Gautam Parai wrote: > > > Congratulations Hanumath! Well deserved :) > > > > Gautam > >

Re: Apache Drill Meetup on Nov 14th!

2018-11-05 Thread Kunal Khatua
Yes, a link of the recording should be available after the meetup. On 11/4/2018 4:38:51 PM, Divya Gehlot wrote: Hi , Can we have the link of recorded video for the audience who cannot attend in person ? Thanks, Divya On Mon, 5 Nov 2018 at 06:46, Pritesh Maker wrote: > Hello, Drillers! > > We ar

Re: Fail to read parquet file with nested column

2018-11-06 Thread Kunal Khatua
Hi Prisdha What do the logs say? Can you share the stack trace from the logs?? Kunal On 11/6/2018 1:58:42 PM, Prisdha Dharma wrote: Hello, The latest Apache Drill works fine with JDBC, JSON, CSV, and simple parquet files. However it fails to read parquet files with nested columns, such as the

Re: Fail to read parquet file with wmi log

2018-11-12 Thread Kunal Khatua
Hi Choi This looks like a bug. Could you please file a bug in the Apache Drill JIRA system? https://issues.apache.org/jira/issues/?jql=project%20%3D%20DRILL%20ORDER%20BY%20key%20DESC%2C%20cf%5B10010%5D%20DESC%2C%20priority%20DESC%2C%20updated%20DESC (Click on the red "Create" button). The data

Re: Status of stored procedure support

2018-11-16 Thread Kunal Khatua
Hi Niels There is currently no one I know who is working on stored procedures. Development is primarily driven by the need for features that give the most benefit to the community of users, and there doesn't seem to be that much demand for this feature. That said, if you think it is a real valu

Re: Drill performance - Waiting time

2018-11-29 Thread Kunal Khatua
Hi Matthias The waiting time for a PARQUET_ROW_GROUP_SCAN operator is the total time that all the fragments took to read the parquet data into memory as Drill's Value Vectors. So, 80 seconds would indicate that the bulk of the time is spent in just getting the data.  If you scroll down to the

Re: SQLPad Support for Drill (JDBC)

2018-12-28 Thread Kunal Khatua
Hi Charles Can you provide steps of where the JDBC driver needs to be located ? I managed to start up the server but I hit a signin message:  Cannot GET /sqlpad/signin I see the following in the server output: [root@kk127 sqlpad]# npm start > sqlpad@2.8.0 start /root/drill/cgivre/sqlpad > concur

Re: Documentation Issue

2019-01-03 Thread Kunal Khatua
Very good point on the 'Added in Drill Version 1.XX' , and I think Bridget's already on it ! However, for sys.functions, it doesn't make sense to carry a min version. The Drill server basically scans all the available functions and exposes them in the sys.functions table. If a function has been

Re: Drill connection using jdbc

2019-01-08 Thread Kunal Khatua
There must be a stack trace in the logs.could you share that? On Tue, Jan 8, 2019, 11:10 PM Tushar Pathare Hello Team, > > We have installed drill 1.12.0 and trying to connect > using a client to the drill cluster > > Our config for connection is > > > > Drill class is : org.a

Re: ACTION: Please Read: Google Summer of Code

2019-01-29 Thread Kunal Khatua
How about a standalone Drill WebServer with support for REST-based Pagination? On 1/29/2019 6:53:43 AM, Charles Givre wrote: Hello all, Although we are experiencing the polar vortex on the East Coast of the US, it’s time to start thinking about the Google Summer of Code. If you have ideas for Dr

Applications of Apache Drill

2019-01-31 Thread Kunal Khatua
Hi everyone As some of you might be aware that the Apache Drill project is into its 5th year of existence and from the rich exchange on the mailing lists, it is clear that it has proven to be of tremendous value to not just enterprise customers using Drill in interesting ways in business-critic

Re: Performace issue

2019-02-08 Thread Kunal Khatua
The memory (heap) would climb as it tries to flatten the JSON data. Have you tried looking at Drill's LateralJoin-Unnest feature? It was meant to address memory issues for some use cases of the FLATTEN operator. On 2/8/2019 5:17:01 AM, PRAVEEN DEVERACHETTY wrote: I am running a query with UNION

Re: Error: SYSTEM ERROR: CompileException: Line 80, Column 24: Assignment conversion not possible from type "java.lang.Object" to type "java.lang.String"

2019-02-09 Thread Kunal Khatua
Hi Krishnanand The mailing lists don't allow attachments. Please provide the Drill version, query, data source and a link to where the relevant part of the log can be downloaded.  The message by itself is not useful, especially without a stacktrace.  On 2/9/2019 11:39:49 AM, Krishnanand Khamb

Re: Performace issue

2019-02-11 Thread Kunal Khatua
getting FLATTENED. example query: 1)first converted into array of json objects 2) flatten to convert into multiple rows select ems.* from (select flatten(t.jdata) as record from (select convert_from('[{"Location":"100","FirstName":"test1"},{"Location&q

Re: Performace issue

2019-02-12 Thread Kunal Khatua
2019 at 11:18 AM Kunal Khatua wrote: > This is a good starting point for understanding LATERAL-UNNEST and how it > compares to the FLATTEN operator. > > https://drill.apache.org/docs/lateral-join/ > > > On 2/11/2019 9:03:42 PM, PRAVEEN DEVERACHETTY wrote: > Thanks Kunal.

Re: Multiple fragments in apache drill

2019-02-13 Thread Kunal Khatua
Hi Hugues The number of fragments is determined by the number of sources (i.e. whether the data can be read in parallel) and the number of estimated rows. CSV and Parquet files are easy to read in parallel, but JSON files are not, because Drill does not know how many JSON documents exist in the

Re: S3 plugin configuration problem cause wrong constructed hostname

2019-02-14 Thread Kunal Khatua
Hi Clark Glad you could resolve the issue. Can you log a JIRA for the issue, and a developer can take a look at it thereafter? Thanks Kunal On 2/14/2019 5:49:58 AM, Updike, Clark wrote: FWIW, this problem was resolved by using the IP address for the S3 target instead of the hostname. No idea w

Anouncing Powered-By-Drill page

2019-02-27 Thread Kunal Khatua
Hi everyone It gives me great pleasure in announcing the launch of the "Powered By Drill" page on the official Apache Drill website : https://drill.apache.org/poweredBy As a start, the page currently has a handful Drill users that shared a short blurb about their usage in production, and one of

Re: Update planner/configuration keys

2019-03-12 Thread Kunal Khatua
Executing an "alter system set param=value" usually persists the value. Not sure if that works for an embedded mode.  Could you try and let us know if that works? On 3/11/2019 11:19:13 AM, PRAVEEN DEVERACHETTY wrote: I am using apache drill on windows platform. My requirement is to udpate the fol

Re: Drill 1.15.0 fails with error while quering Parquet 2.0 file

2019-03-12 Thread Kunal Khatua
Hi Denis You seem to be trying to read a Parquet 2.0 format file with a Parquet 1.10 reader that comes with Drill. Is there a specific reason you are using version 2.0 ? ~ Kunal On 3/11/2019 10:13:39 AM, Denis Dudinski wrote: Hello, I have a parquet 2.0 file which contains serialised avro rec

Re: Update planner/configuration keys

2019-03-12 Thread Kunal Khatua
ink these scripts > only on the drill command prompt right? > > Thanks, > Praveen > > On Tue, Mar 12, 2019 at 1:02 PM Kunal Khatua wrote: > > > Executing an "alter system set param=value" usually persists the value. > > Not sure if that works for an embedd

Re: Json Complex array issue only on HDF, local storage is fine

2019-03-13 Thread Kunal Khatua
Hi Rob This is very weird ! Is that the entire JSON? I'm wondering if the JSON document by any chance has a different end-of-line character that *might* be causing this. Windows uses CarriageReturn+LineFeed, while Linux (HDFS) has only Linefeed and MacOS using CarriageReturn. Linux has a dos2u

Re: Embeded drill

2019-03-25 Thread Kunal Khatua
Hi Praveen THe mailing lists dont allow attachments to be sent through. What ports are you trying to change? ~ Kunal On 3/25/2019 9:59:29 AM, PRAVEEN DEVERACHETTY wrote: I have a question regarding the ports used in embeded drill. Is there anyway to disable these ports in embeded drill option.

Re: Embeded drill

2019-03-25 Thread Kunal Khatua
e not opened. Drill also opening lot of ports during execution. I am not sure what ports are they On Mon, Mar 25, 2019, 22:46 Kunal Khatua wrote: > Hi Praveen > > THe mailing lists dont allow attachments to be sent through. What ports > are you trying to change? > > ~ Kunal &g

Re: [ANNOUNCE] New PMC member: Sorabh Hamirwasia

2019-04-05 Thread Kunal Khatua
Congratulations! Very well deserved!  On 4/5/2019 11:02:58 AM, Boaz Ben-Zvi wrote: Congratulation Sorabh - welcome to the Project Management Committee !! On Fri, Apr 5, 2019 at 10:58 AM Abhishek Ravi wrote: > Congratulations Sorabh! Well deserved! > > On Fri, Apr 5, 2019 at 10:49 AM hanu mapr

Re: Drill for Data Virtualization

2019-04-10 Thread Kunal Khatua
Hi Sarnath >From what I understand by your description, you are looking to see if Drill >can push down the GROUP BY clause to the underlying JDBC sources A and B. Unfortunately, Drill does not support pushdown for the JDBC storage plugin as yet. That said, even if the feature existed, by design

Re: Drill for Data Virtualization

2019-04-11 Thread Kunal Khatua
ts pretty recent - Dec 2018). Thanks for your time, Best, Sarnath On Thu, Apr 11, 2019 at 1:54 AM Kunal Khatua wrote: > Hi Sarnath > > From what I understand by your description, you are looking to see if > Drill can push down the GROUP BY clause to the underlying JDBC sources A >

Re: How to store/export personal settings of Drill

2019-04-11 Thread Kunal Khatua
That's very simple! You can query the table - "sys.options" and use CTAS command. The simplest approach would be the following steps: 1. Log into a SQLLine session 2. Tinker with all the settings that you are trying, using "ALTER SESSION". This ensures that nothing permanently persists. 3. Run t

Re: Drill for Data Virtualization

2019-04-11 Thread Kunal Khatua
ement for that particular sourceAnd we try to union them and group again...This way, explicitly making up the Query do the pushdown That's the idea you are referring to. Right!?? Kunal Khatua: That is correct. Worth a try. Start with querying the views individually to see if the pushdown

Re: Drill for Data Virtualization

2019-04-17 Thread Kunal Khatua
ollowing > it for quite some time. Thanks!! > Best, > Sarnath > > On Fri, Apr 12, 2019, 02:28 Kunal Khatua wrote: > >> On 4/11/2019 12:39:24 PM, Sarnath K wrote: >> Thank you Kunal. >> >> >>>You could try creating views for each source and then doi

Re: Blocker on drill upgrade path

2019-04-19 Thread Kunal Khatua
(Replying on the Drill user list)  This is odd. The CP storage plugin is inbuilt with Drill and that hasn't changed. 1.15 by itself works fine.  What is the error you are seeing, Nitin?  On 4/18/2019 10:58:48 PM, Nitin Pawar wrote: Hi, We are trying to upgrade drill from 1.13 to 1.15 followi

RE: Strange join table criteria in apache drill

2019-04-23 Thread Kunal Khatua
(Adding this to the user mailing list for future reference and see if anyone else has alternative solutions) Hi Charles What you pointed out is the same issue as the mail I shared regarding the GROUP BY clause. There seems to be a limitation in the Calcite planner w.r.t. the JDBC storage plugi

Re: [VOTE] Apache Drill Release 1.16.0 - RC1

2019-04-24 Thread Kunal Khatua
Downloaded the tarball and tried it in embedded mode.  Ran simple join queries and interacted with the WebUI.  Issues confirmed were DRILL-7192 and DRILL-7203.  I'm unable to repro DRILL-7201 and DRILL-7202, though I have a fix for the latter. Will work with Arina to identify repro steps. None

Re: [VOTE] Apache Drill Release 1.16.0 - RC1

2019-04-25 Thread Kunal Khatua
e column (i.e. all the other keys > > as well) was marked as "interesting"; but this may be "by design", as > > the refresh granularity is the whole column. > > > > Also noticed the sys.version issue (DRILL-7208 > > ) - should be minor as >

Re: Struggling with mssql schema names

2019-04-28 Thread Kunal Khatua
Did you try using the accent quotes/back ticks (on the top left neighbouring the TAB key) to enclose the schema and table names?  You might have to experiment with something like `ABC.PQR`.`XYZ` or `ABC`.`PQR.XYZ`. This is how it is done for resolving paths across, so it shoudl work the same f

Re: [VOTE] Apache Drill Release 1.16.0 - RC2

2019-04-30 Thread Kunal Khatua
Ran manual tests with random queries, trying out the UI and running joins on small tables.  HOCON export of storage plugins does not actually export in HOCON format, but that is not a blocker. +1 (binding) ~ Kunal  On 4/30/2019 4:53:48 AM, Arina Yelchiyeva wrote: Downloaded binary tarball an

Re: Serious performance problem in ctas json to parquet writing

2019-05-06 Thread Kunal Khatua
Hi Mehran What does the full plan look like?You seem to be missing the SCAN operator. Can you post the plans for 1.13 as well? It isn't clear from the table if there is indeed a problem, since you've not shared the query that created this plan. Also, there have been changes gone in, such as bat

RE: WELCOME to user@drill.apache.org

2019-05-15 Thread Kunal Khatua
It could be related to Hive versions. You can try an older version of Drill to verify this... or upgrade to Hive 2.x which is not expected to be fully backward compatible with Hive 1.x (hence the major version change). Also, if I would suggest you experiment with the back ticks to see if that m

Re: image

2019-05-15 Thread Kunal Khatua
Hi The mailing list does not allow you to send image attachments. Can you share your issue (preferably) in English? Thanks Kunal On 5/15/2019 10:00:14 PM, 高 wrote: 您好,DRILL团队     我在使用drill发现了两个问题,如下:                    

Drill Meetup @ Santa Clara (Wednesday)

2019-05-21 Thread Kunal Khatua
Hi everyone As some of you know, we have a Drill Meetup in MapR's headquarters at Santa Clara, CA. Here is the link to the sign up sheet: https://www.meetup.com/Bay-Area-Apache-Drill-User-Group/events/260840985/ Folks from Alibaba will be presenting how they are leveraging Drill on top of large

Re: Superset and Drill

2019-06-02 Thread Kunal Khatua
Fantastic work! I'll try to promote this on the twitter handle as well. Thank you!  ~ Kunal On 6/2/2019 7:34:48 AM, Charles Givre wrote: Hello Everyone, I wanted to send this note to the Drill aliases but as of this weekend, the Drill integration with Superset is complete and merged. If you ha

Re: Schema support in storage and format plugins

2019-06-03 Thread Kunal Khatua
Hi Paul Adding this to the user mailing list as well, to broaden the reach of this thread. ~ Kunal On 5/27/2019 6:23:50 PM, Paul Rogers wrote: Hi All, Drill 1.16 introduced the the "provided schema" mechanism to help you query the kind of messy files found in the real world. Arina and Bridget

Re: Heap memory - leak

2019-06-25 Thread Kunal Khatua
Yes. The reason is that "0% of 0GB" basically means 0% of the maximum direct memory claimed so far.  Drill will consume some system memory for use in Direct and can (potentially) return it as well. In your case, the Direct memory appears to be barely used. So, it is very likely that the Drillbi

Re: [ANNOUNCE] New Committer: Bohdan Kazydub

2019-07-18 Thread Kunal Khatua
Congratulations, Bohdan! On 7/16/2019 2:21:14 PM, Robert Hou wrote: Congratulations, Bohdan. Thanks for contributing to Drill! --Robert On Tue, Jul 16, 2019 at 11:50 AM hanu mapr wrote: > Congratulations Bohdan! > > On Tue, Jul 16, 2019 at 9:30 AM Gautam Parai wrote: > > > Congratulations Bohda

Re: [ANNOUNCE] New Committer: Igor Guzenko

2019-07-23 Thread Kunal Khatua
Congratulations, Igor! On Tue, Jul 23, 2019, 8:13 AM Aman Sinha wrote: > Congratulations Igor and thanks for your contributions to Drill ! > > On Tue, Jul 23, 2019 at 3:33 AM Anton Gozhiy wrote: > > > Congratulations Igor, well deserved! > > > > On Tue, Jul 23, 2019, 12:31 Volodymyr Vysotskyi

Re: [ANNOUNCE] New Committer: Anton Gozhyi

2019-07-29 Thread Kunal Khatua
Congratulations, Anton!! Very well deserved! On 7/29/2019 9:11:30 AM, Vitalii Diravka wrote: My congratulations Anton! You deserved it! Kind regards Vitalii On Mon, Jul 29, 2019 at 1:13 PM Arina Ielchiieva wrote: > Congratulations Anton! Thanks for your contributions. > > Kind regards, > Arina

Re: [ANNOUNCE] New PMC Chair of Apache Drill

2019-08-22 Thread Kunal Khatua
Congratulations, Charles!

Re: [ANNOUNCE] New Committer: Denys Ordynskiy

2020-01-02 Thread Kunal Khatua
Congratulations, Denys! ~ Kunal On Wed, Jan 1, 2020 at 12:19 AM Pritesh Maker wrote: > Congrats, Denys! > > Pritesh > > > From: Igor Guzenko > Sent: Tuesday, December 31, 2019 2:39 PM > To: dev > Cc: user > Subject: Re: [ANNOUNCE] New Committer: Denys Ordynskiy

<    1   2   3