Re: maprdb hbase drill does not get any result

If Drill doesnt show any results, and no errors... it's a permissions issue. By design, MapRDB will not reveal anything about the requested table due to security reasons. You can try running the following on all of the tables you wish to query.  maprcli table cf edit -path ${tblPath} -cfname ${c

Re: Re: Hbase SQL

of any variables, properties, etc... should have atleast one leading non-numeric character. You might try having the column name defined in UTF-8 for readability and ease in querying. Hope that helps. Kunal Khatua Engineering (Performance) [image: MapR] <http://www.mapr.com> www.mapr.c

Re: Question about querying array fields in parquet files

t;post a set of the sample records as well? Thanks Kunal  Kunal Khatua Engineering [ MapR] [http://www.mapr.com/] www.mapr.com [http://www.mapr.com/] On Thu 23-Jun-2016 1:52:48 PM, David Kincaid wrote: I'm very new to Drill and just learning how everything works. I had a question about a query w

Re: Speed up Drill

If you are using the JDBC plugin to connect to SqlServer, which is a community storage plugin, performance might be sub-optimal.  If you are trying to do a table scan, you also have the possibility that the data being streamed is very large. Your terminal might be the bottleneck for having to d

Re: Cannot load Parquet files created with parquet-cpp in Drill

Hi Uwe I believe you're using the latest Apache Drill 1.8.0. From a quick look at the stack trace, it appears to be a potential bug on Drill's interpretation of dictionary encoded data.  One way to verify that your C++ implementation of Parquet is correct would be to have your generated data w

Re: Date Formatting Question

Hi Charles, There isn't a single function that would convert to the format you want, since that would be a user-specific format and not native to the internal Drill representation of the date format.  Treating the output format of 'mm/dd/' as a string is a solution, using the date-time fun

Re: drill with hive snappy

If you are specifically referring to snappy-compressed Parquet data, yes.  Drill can also leverage Hive's own libraries to consume Hive-generated data (for e.g. ORC format) that might not be directly consumed by Drill. On Mon 24-Oct-2016 9:54:49 AM, Rainday Chu(初雨) wrote: Hi guys, We are very

Re: How to set output file size for parquet to csv conversions?

Laura There isn't a way to limit the size of individual files. However, there might be a way you can get around this.  One option is to serially convert each Parquet file into a CSV file separately. Typically, a fragment within Drill will generate this CSV file. That might generate 12 CSV file

Re: Last Column showing blank in csv file

There is a dos2unix utility for Linux that allows you to substitute the multichar newline with the single char newline. For Windows, you can use either a similar util on SourceForge or the CygUtils (part of the Cygwin shell, I believe) to achieve the same conversion.  In the meanwhile, like Abh

Re: how to control the number of profiles on webui

Currently, the value is hard-coded to a limit of 100. https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/ProfileResources.java If you are building Drill from source, you could fork off the Drill repo and modify the value for MAX_PRO

Re: UDF parameters

Have you tried looking at the String manipulation function - CONCATE function as a reference to achieve your goal? Kunal Khatua From: Muhammad Gelbana Sent: Monday, January 23, 2017 1:18:21 AM To: user@drill.apache.org Subject: UDF parameters I need to create

Re: Explain Plan for Parquet data is taking a lot of timre

his should increase parallelization and the utilization of all the cores by Drill. Kunal Khatua Engineering [MapR]<http://www.mapr.com/> www.mapr.com<http://www.mapr.com/> From: Jeena Vinod Sent: Tuesday, February 28, 2017 12:24:52 PM To: user@dr

Re: Discussion: Comments in Drill Views

+1 I this this can be very useful. The only worry is of someone abusing it, so we probably should have a limit on the size of this? Not sure else it could be exposed and consumed. Kunal Khatua Engineering [MapR]<http://www.mapr.com/> www.mapr.com<http://www

Re: Discussion: Comments in Drill Views

uming we'd also probably want something like a DESCRIBE VIEW ... to be enhanced to something like DESCRIBE VIEW WITH COMMENTARY ... A 1KB field is quite generous IMHO. That's more than 7 tweets to describe something ! [?] Kunal Khatua From: Ted Dunni

Re: Configuring Drill Memory Usage under Windows

I've not tried running Drill embedded on Windows, but you can try checking the parameters passed to the running Drill JVM to validate that the settings were picked up ? Kunal Khatua Engineering [MapR]<http://www.mapr.com/> www.mapr.com<http:

Re: Configuring Drill Memory Usage under Windows

I've not tried running Drill embedded on Windows, but you can try checking the parameters passed to the running Drill JVM to validate that the settings were picked up ? Kunal Khatua Engineering [MapR]<http://www.mapr.com/> www.mapr.com<http:

Re: Minimise query plan time for dfs plugin for local file system on tsv file

re.dfs.FileSelection - FileSelection.getStatuses() took 0 ms, numFiles: 1 More than 30 secs is unaccounted for. Can you turn on the root logger to be at the debug level and retry the explain plan? Kunal Khatua From: rahul challapalli Sent: Tuesday, March 7, 2017 5:24:43 AM To: user S

Re: Query on .gz.parquet files

Is this a Parquet file? It looks more like a JSON document. What is the schema description published by the parquet-tools? From: PROJJWAL SAHA Sent: Thursday, March 9, 2017 4:36:06 AM To: user@drill.apache.org Subject: Query on .gz.parquet files All, one ques

Re: Display of query result using command line

The SQLLine shell is a CLI utility to allow people to explore big data.. but not designed (intentionally) to print 3 million rows! The utiity tries to format the data, and has limited internal buffers. You probably want to use a simple JDBC program to achieve the purpose of printing out the re

Re: [ANNOUNCE] Apache Drill 1.10.0 Released

Hi Bob Thanks for the positive acknowledgement of our efforts. We're also interested in understanding what reformatting (I'm guessing you mean code refactoring?) did you do for workarounds. As an open source project, we look forward to contributions from the community in solving problems that

Re: Quoting queries

What is the client machine from where you're issuing the query? Is it Windows? Drill 1.10 has a fix for Windows wildcard: https://issues.apache.org/jira/browse/DRILL-4812 Can you try with that? Kunal Khatua Engineering [1490734684477_mapr.png] www.mapr.com<http://www.

Re: JDBC disconnections over remote networks

ats, and the server closes the connection because that thread unable to send heartbeats (blocked). This could result in the JDBC disconnection. You could suffix a LIMIT to the query to identify the 'sweet spot' where disconnections don't happen. Kunal Khatua _

RE: Quoting queries

ies BTW I tried with the 1.10 version on both sides (server and client) and I still get the same error. -Ursprüngliche Nachricht- Von: Kunal Khatua [mailto:kkha...@mapr.com] Gesendet: Donnerstag, 30. März 2017 20:04 An: user Betreff: Re: Quoting queries What is the client machine from wh

RE: Struggling to run in distributed mode

What is the error for the collocated Drillbit instance ? Did you apply the "DRILL_HOST_NAME" to the collocated Drillbit's drill-env.sh file as well? One quick test is to try if the ZK port is accessible locally, but not using the loopback IP (i.e. 127.0.0.1 ). Telnet is a good way to verify that

RE: Struggling to run in distributed mode

ns did not mention it? Also, why cannot my machine telnet to itself? Michael Knapp On 3/31/17, 3:05 PM, "Kunal Khatua" wrote: What is the error for the collocated Drillbit instance ? Did you apply the "DRILL_HOST_NAME" to the collocated Drillbit's drill-env.sh fi

Re: Quoting queries

_____ From: Kunal Khatua Sent: Friday, March 31, 2017 11:58:56 AM To: user@drill.apache.org Subject: RE: Quoting queries This seems to be a windows JDBC client specific problem. Using SQuirreL, I tried with this on Windows while accessing a Linux server: select * from `nation_json/a/1/*/nation.js

Re: Struggling to run in distributed mode

me=ip-10-XXX-YYY-ZZZ..com. meaning it was not localhost. On 3/31/17, 4:36 PM, "Kunal Khatua" wrote: "DrillbitStartupException: Drillbit is disallowed to bind to loopback address in distributed mode" I believe this is because you are using localhost / 127.0.

Re: querying S3 bucket root dir

You'll need to log into https://issues.apache.org/jira/browse/ and click on red 'Create' button on the top to create an issue. Mark the Project as "Apache Drill" and Issue type as "Bug" From: Alon Principal Sent: Monday, April 17, 2017 12:08:46 PM To: user@d

Re: Windows system Initiation Error

I am able to log into a remote Linux-based Drillbit using SQLLine with no issues. C:\drill\drill-1.10.0-SNAPSHOT\bin>sqlline.bat -u "jdbc:drill:zk=10.10.120.160:2181" DRILL_ARGS - " -u jdbc:drill:zk=10.10.120.160:2181" HADOOP_HOME not detected... HBASE_HOME not detected... Calculating Drill cla

Re: [Drill 1.10.0] : Memory was leaked by query

Could you also share the profiles for the failed queries as well? Thanks Kunal From: Padma Penumarthy Sent: Tuesday, April 18, 2017 7:18:08 AM To: user@drill.apache.org Cc: d...@drill.apache.org Subject: Re: [Drill 1.10.0] : Memory was leaked by query Seems li

Re: Struggling to run in distributed mode

rst email: I looked in the drillbit.log file, it has: host.name=ip-10-XXX-YYY-ZZZ..com. meaning it was not localhost. On 3/31/17, 4:36 PM, "Kunal Khatua" wrote: "DrillbitStartupException: Drillbit is disallowed to bind to loopback address in distributed mode"

Re: JDBC SQL parse error on CTAS

I'm not sure if you can do a select * from (CTAS ..) The output of CTAS really is only a metric of the rows that got written by the various fragments. Are you able to run the query in SQLLine? The parse error does a pretty good job of pointing out where the query appears to have broken the par

Re: Quoting queries

Hi David I tried the queries on SQuirreL and everything following the asterisk in the 'table' name is truncated when the query is submitted by it. However, when I tried another tool: DBeaver (http://dbeaver.jkiss.org/) it worked! Like SQuirreL, the DBeaver client's SQL Editor also shows the

Re: multiple users and passwords

Have you had a look at this link? https://drill.apache.org/docs/configuring-user-authentication Configuring User Authentication - Apache Drill drill.apache.org Authentication is the process of establishing confidence of authenticity

Re: Quoting queries

maximize developer productivity. Together, powerful static code analysis and ergonomic design make ... that don't work. Maybe it would be useful to contact somebody on the team of either. I can open a ticket at intellij if you want as we have paid support. Regards, David -----Ursprüngl

Re: Drill query are stuck in ENQUEUED mode

hile the others completed. So you should be able to find that easily. Kunal Khatua Engineering [1490734684477_mapr.png] www.mapr.com<http://www.mapr.com/> From: jasbir.s...@accenture.com Sent: Wednesday, May 3, 2017 9:37:44 AM To: user@drill.apache.org Cc

Re: Quoting queries

lower part of the panel. -- This worked for me. I'm presuming there *might* be something similar for other tools as well. I'll close the ticket with SQuirreL in the meanwhile. Kunal ________ F

Re: Drill query are stuck in ENQUEUED mode

e know how to resolve query stuck issue as it is hampering products performance. Regards, Jasbir Singh -Original Message----- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: Wednesday, May 03, 2017 10:43 PM To: user@drill.apache.org Cc: Kothari, Maneesh ; Kumar, H. P. Subject: Re: Drill

Re: In-memory cache in Drill

Drill does not cache data in memory because it introduces the risk of dealing with stale data when working with data at a large scale. If you want to avoid hitting the actual storage repeatedly, one option is to use the 'create temp table ' feature (https://drill.apache.org/docs/create-tempora

Re: In-memory cache in Drill

the temporary tables. Sincerely, Michael Shtelma On Wed, May 10, 2017 at 6:30 PM, Kunal Khatua wrote: > Drill does not cache data in memory because it introduces the risk of dealing > with stale data when working with data at a large scale. > > > If you want to avoid hitting the

Re: In-memory cache in Drill

y, > Michael Shtelma > > > On Wed, May 10, 2017 at 6:30 PM, Kunal Khatua wrote: > > Drill does not cache data in memory because it introduces the risk of > dealing with stale data when working with data at a large scale. > > > > > > If you want to avoid hittin

Re: Not Able to suscribe

Did this not work? https://drill.apache.org/mailinglists/ From: Arvind Sharma Sent: Monday, May 15, 2017 12:00:33 AM To: user@drill.apache.org Subject: Not Able to suscribe

Re: Drill embedded mode

Is there a reason you want to use Drill in Embedded mode and not as a standalone server? You can always use a screen session and start up Drill embedded mode and then detach that session. From: Selvarajan Thangavel Sent: Sunday, May 14, 2017 7:31:58 AM To: use

Re: Run Multiple Queries as a Single query Apache Drill

You could try to set the value through the OPTIONS tab on the top right. This will be a permanent 'system' level setting and not a temporary 'session'-level setting. You can have your API also do this using the 'alter system set ...' to achieve the same end goal. ___

Re: error in mac OS X driver installer instructions

Looks like you're right about the typo. We'll have it corrected. Thanks, Jonathan! ~ Kunal From: Jonathan Snyder Sent: Friday, May 19, 2017 1:30:17 PM To: user@drill.apache.org Subject: error in mac OS X driver installer instructions H

Re: Parquet filter pushdown and string fields that use dictionary encoding

Even though filter pushdown is supported in Drill, it is limited to pushing down of numeric values including dates. We do not support pushdown of varchar because of this bug in the parquet library: https://issues.apache.org/jira/browse/PARQUET-686 The issue of correctnes

Re: Parquet filter pushdown and string fields that use dictionary encoding

nary encoding Thank you Kunal. Kan you please explain to me why min/max values would be relevant for dictionary encoded fields? (I think I may be completely misunderstanding how they work) Regards, -Stefán On Wed, May 31, 2017 at 5:55 PM, Kunal Khatua wrote: > Even though filter pushdo

Re: [External] Re: UNORDERED_RECEIVER taking 70% of query time

Hi Jasbir I don't think the Apache mailing lists allows you to send attachments, except may be text files. (The txt file made it through). In your Operator Profile, you'll see two columns... %Fragment Time and %QueryTime Taking your mouse over those table headers should show you a descriptio

Re: [External] Re: UNORDERED_RECEIVER taking 70% of query time

we can try reducing it? Regards, Jasbir singh -Original Message- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: Saturday, June 03, 2017 12:00 AM To: user@drill.apache.org; d...@drill.apache.org Cc: Kothari, Maneesh ; Sareen, Nitin A. ; Kumar, H. P. Subject: Re: [E

Re: Column alias are ignored when Storage Plugin is enabled

It could be related to these as well: https://issues.apache.org/jira/browse/DRILL-5537 https://issues.apache.org/jira/browse/DRILL-5538 Please go ahead and file a bug. If it is related, they'll be linked and resolved together. ~ Kunal From: Rahul Raj Sent:

Re: Increasing store.parquet.block-size

Shuporno There are some interesting problems when using Parquet files > 2GB on HDFS. If I'm not mistaken, the HDFS APIs that allow you to read offsets (oddly enough) returns an int value. Large Parquet blocksize also means you'll end up having the file span across multiple HDFS blocks, and th

Re: Increasing store.parquet.block-size

csv source files can be quite huge (around 10GB). So, is there a way to overcome this and create one parquet file or do I have to go ahead with multiple parquet files? On 09-Jun-2017 11:04 PM, "Kunal Khatua" wrote: > Shuporno > > > There are some interesting problems when u

Re: Increasing store.parquet.block-size

d the file later? On 09-Jun-2017 11:28 PM, "Kunal Khatua" wrote: > > If you're storing this in S3... you might want to selectively read the > files as well. > > > I'm only speculating, but if you want to download the data, downloading as > a queue of file

RE: Using Apache Drill with AirBnB SuperSet

The Superset project looks pretty neat. Didn't realize that there is also a Python Driver for Drill. I'd think that would be useful too. -Original Message- From: John Omernik [mailto:j...@omernik.com] Sent: Wednesday, June 14, 2017 3:45 AM To: user Subject: Using Apache Drill with Ai

RE: JDBC help

You need to use the org.apache.drill.jdbc.Driver class. The drill-jdbc-all dependency should be enough. Are you trying to connect to an HDFS cluster's zookeeper? -Original Message- From: Aspen Hsu [mailto:a...@xactlycorp.com] Sent: Wednesday, June 14, 2017 12:07 PM To: user@drill.apac

RE: Apache Drill Question

Not familiar with SSHFS or GlusterFS specs, but It should, in theory, work out of the box. You can start off Drill with having the underlying storage plugins talk to a localFS. I'm presuming SSHFS / GlusterFS can expose the files through a local NFS-like mount. However, if your three nodes al

RE: SIGSEGV error - StubRoutines::jlong_disjoint_arraycopy

Nope.. not seen this before. Can you share more details of the log messages, etc? The problem might have to do with the JSON files being very large... because the segmentation fault that triggered the JVM (Drillbit) crash hints at that during the write of the Parquet files. I take it you are

Re: SIGSEGV error - StubRoutines::jlong_disjoint_arraycopy

- StubRoutines::jlong_disjoint_arraycopy Yeah. It only crashes on the larger JSON files. Reworking my python script to use hdfs.tmp instead of dfs.tmp now.. -Original Message----- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: Thursday, June 15, 2017 10:52 AM To: user@drill.apache.org Subject: RE

RE: Apache Drill C++ Client Binary Versioning Information

I don't think the mailing lists allows attachments to go through. Can you share via some free online utility like https://imgur.com/upload From: Robert Wu [mailto:r...@magnitude.com] Sent: Tuesday, June 13, 2017 12:07 PM To: user@drill.apache.org Subject: Apache Drill C++ Client Binary Versioni

RE: Apache Drill C++ Client Binary Versioning Information

I think you missed sharing the Apache JIRA :) From: Robert Wu [mailto:r...@magnitude.com] Sent: Thursday, June 22, 2017 5:19 PM To: user@drill.apache.org Subject: RE: Apache Drill C++ Client Binary Versioning Information Oh sorry, looks like attachment is not going through. I've captured the info

RE: Rest API - Need to Improve

Should we look at some well-established products that have a good set of such APIs for guidance? That will ensure that we atleast identify the most relevant APIs. -Original Message- From: John Omernik [mailto:j...@omernik.com] Sent: Wednesday, July 12, 2017 6:12 AM To: user Subject: Re

RE: Error when timestamp IN clause contains more elements

This works for 19 elements because the IN clause will do a filter pushdown to the source for up to 19 elements. 20 or more... it resorts to using a JOIN, in which case it finds data type mismatch. It is a good practice (for readability purposes) to do casts in such cases. Alternatively (not re

RE: drill error connecting to Hbase

This means that the connectivity with ZK appears to be working. What are the HBase, ZK and Hadoop versions that you are working with? I presume that the student table is otherwise accessible. -Original Message- From: Shai Shapira [mailto:shai.shap...@amdocs.com] Sent: Sunday, July 23,

RE: drill error connecting to Hbase

error connecting to Hbase It is CDH 5.8.2 I believe it is reliable versions, isn't it? Thanks, Shai -Original Message----- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: Monday, July 24, 2017 8:50 AM To: user@drill.apache.org Subject: RE: drill error connecting to Hbase This mean

RE: regex replace in string

Here is the reference mail for the release candidate of Drill 1.11.0 -Original Message- From: Arina Yelchiyeva [mailto:arina.yelchiy...@gmail.com] Sent: Tuesday, July 25, 2017 3:36 AM To: d...@drill.apache.org Subject: [VOTE] Release Apache Drill 1.11.0 - rc0 Hi all, I'd like to propose

RE: Drill performance tuning parquet

You haven't specified what kind of query are you running. The Async Parquet Reader tuning should be more than sufficient in your usecase, since you seem to be only processing 3 files. The feature introduces a small fixed pool of threads that are responsible for the actual fetching of bytes from

RE: Drill performance tuning parquet

o the approach would be to tune it for the entire set (which means you will end up trading off the various parameters) and then scale out. (Scale out is not cheap). Thanks, Saurabh On Thu, Jul 27, 2017 at 1:37 PM, Kunal Khatua wrote: > You haven't specified what kind of query are

RE: Drill performance tuning parquet

20 > > > Dan Holmes | Revenue Analytics, Inc. > Direct: 770.859.1255 > www.revenueanalytics.com > > -Original Message- > From: Kunal Khatua [mailto:kkha...@mapr.com] > Sent: Friday, July 28, 2017 2:38 AM > To: user@drill.apache.org > Subject: RE: Drill performance t

RE: delimiter in column values

I think you need quotes around the single word datasets as well, because the quotes act as String delimiters and help in indicating the start and end of a String. Is there a reason why the single word strings cannot be in quotes as well? -Original Message- From: Divya Gehlot [mailto:div

RE: delimiter in column values

t;colonedata4" "coltwodata4" "-33.8724176" "151.2067579" "" "colonedata5" "coltwodata5" "" "" "" "This col6 data" "coltwodata6" "-33.869732" "151.203" "This co

RE: delimiter in column values

You could just try to have the headers in a single line too... emulating the structure that the rest of the data follows. -Original Message- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: Tuesday, August 01, 2017 9:38 PM To: user@drill.apache.org Subject: RE: delimiter in column

RE: pass parameters to Drill query in file

If you just want to run the query with the parameters like "alter session", you can write those session-altering SQL as the initial set of queries before the actual query in the same file. Then run something like this: /opt /drill/apache-drill-1.11.0/bin/sqlline -u "jdbc:drill:zk=local" --force

RE: delimiter in column values

will start as > VarChar. You can convert to a numeric type in the query. > > Example using your data: > > Column1,Column2,Column3,Column4,Column5 > colonedata1,coltwodata1,-35.924476,138.5987123, > colonedata2,coltwodata2,-27.4372536,153.0304583,137 > > Note that if col

Re: delimiter in column values

select `col_Column1` from `dfs`.`installedsoftwares/ApacheDrill/apache-drill-1.10.0.tar/apache-drill-1.10.0/sample-data/sample_data.csv`; ++ | col_Column1 | ++ | "colonedata1" | | "colonedata2" | | "colonedata3" | | "colonedata4

RE: drill error connecting to Hbase

M To: user@drill.apache.org; Kunal Khatua Subject: RE: drill error connecting to Hbase Attached the relevant part from the sqlline.log, Hope it helps Thanks, Shai -Original Message- From: Shai Shapira Sent: Thursday, August 03, 2017 11:04 AM To: kkha...@mapr.com Cc: user@drill.apache.org Subject:

RE: Drill JDBC connection on windows in embedded node

+1 to Andries' comment. When you start Drill in Embedded mode, you are bringing up a single standalone Drillbit without any Zookeeper that would otherwise allow discovery of other Drillbits in the cluster. Starting in embedded mode is done via SQLLine because in a practical usage, a single D

RE: unable to run drill sql in shell script

You seem to be missing the jars in the classpath. Notice the line before the error... "Calculating Drill classpath" What you probably need to do is to navigate to your installation's bin directory and launch from there. -Original Message- From: Divya Gehlot [mailto:divya.htco...@gmail

RE: pass parameters to Drill query in file

ed Thanks, Divya On 3 August 2017 at 12:30, Kunal Khatua wrote: > If you just want to run the query with the parameters like "alter > session", you can write those session-altering SQL as the initial set > of queries before the actual query in the same file. > > The

RE: delimiter in column values

using Drill 1.11.0 3. There are table functions in Drill that guide it with additional inputs on how to manage the preparation of the table. Can you please share the link ? Thanks, Divya On 3 August 2017 at 21:06, Kunal Khatua wrote: > A couple of things... > > 1. Your delimiter is a pi

RE: set up Apache Drill on Windows server in distributed mode

Most people have used Apache Drill on Windows primarily in Embedded mode because no one appears to have tried for more than 1 Drillbit. That said, you should be able to run Apache Drill in a distributed mode as well, since it is Java-based and would not need to rely on anything more than a Zoo

RE: set up Apache Drill on Windows server in distributed mode

dbc:drill:zk=NameNode,DataNode1,DataNode2 (The filename, directory name, or vol ume label syntax is incorrect) apache drill 1.11.0 "got drill?" sqlline> Note : Removed the actual IP for security purpose. Appreciate the help . Thanks, Divya On 11 August 2017 at 08:20, Kunal Kh

RE: stop a long running query

Are you attempting to cancel the query through the Drill WebUI or some web interface for Tableau? Is there no option within Tableau to cancel a query? Drill supports query cancellation in Drill, so you should be able to a trigger cancellation that way. To cancel a query which was launched by so

RE: Merge and save parquet files in Drill

If you are creating lot of small files within each partition, that is because different writers write a file for each partition. One workaround would be to reduce the number of writer fragments. We can achieve that by setting the parameter to a lower value, so that you generate larger files. Th

RE: Unable to SELECT from parquet file with Hadoop 2.7.4

Interesting. I'm presuming this works if the parquet file is in a directory, right? Was Drill built with Hadoop 2.7.4 dependencies or did you use the default 2.7.1 that is there in the POM.XML ? A workaround for now would be to query on an enclosing directory, until someone looks at the issue i

RE: Unable to SELECT from parquet file with Hadoop 2.7.4

t pom.xml) Hadoop 2.7.4 with Drill 1.11 (with -Dhadoop.version=2.7.4) Hadoop 2.8.0 with Drill 1.11 (default pom.xml) Hadoop 3.0.0-alpha4 with Drill 1.11 (default pom.xml) Kind regards, Michele On Fri, Aug 18, 2017 at 8:06 PM, Kunal Khatua wrote: > Interesting. I'm presuming this works if

RE: Unable to SELECT from parquet file with Hadoop 2.7.4

hanks Kunal -Original Message- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: Friday, August 18, 2017 1:07 PM To: user@drill.apache.org Subject: RE: Unable to SELECT from parquet file with Hadoop 2.7.4 Glad that worked ! Not sure why the APIs in 2.7.4+ are not backward compatible, b

RE: Drill selects column with the same name of a different table

Could you share the profile ( *.sys.drill file or the http://:8047/profiles/.json ) ? This might be a bug with the JDBC Storage plugin. A quick way to validate this would be to have the similar data as 2 text/parquet tables and have Drill read from that. If we don't see an issue, then it is m

RE: Drill Profile page takes too much time to load

This might be because that installation is possibly hosting a lot of Drill profiles. Do you know how many profiles you have residing in the underlying persistent store? When a query is executed, the Foreman Drillbit (i.e. the node from where the query is submitted) writes the final profile to t

RE: Problems using Postgres datasource

I'm not very familiar with the details of Postgres, but I do so see people occassionally asking about it Have you checked the mailing list archives? You might find your answers there. -Original Message- From: Gonzalo Ortiz Jaureguizar [mailto:golthir...@gmail.com] Sent: Friday, Septembe

RE: Spark exception crashing application

The exceptions/error (NoClassDefFoundError) appear to indicate that there is a possible mismatch between the library versions of Spark within Drill and the platform you are running on.. Can you start by verifying the versions of the Spark libraries you have and then try to build Drill (edit the

RE: Apache DRILL v1.11.0 handling Postgres citext columns with Inconsistency

It's odd that adding just a term_count column is causing an error but the other 2 columns (created, updated) don't seem to be... and gets resolved on removing the cast. Can you provide the stack trace and error message? Also, what are the data types for the other columns? -Original Messag

Re: User client timeout with results > 2M rows

Do you know in how much time does this timeout occur? There might be some tuning needed to increase a timeout. Also, I think this (S3 specifically) has been seen before... So you might find a solution within the mailing list archives. Did you try looking there? From: Alan Höng Sent: Wednesday

RE: User client timeout with results > 2M rows

th results > 2M rows Yes it takes about 2-3min for the timeout to appear the query itself should finish in that time. The files are not that big for debugging. I have, but I couldn't find anything relevant or helpful in my situation so far. On Wed, 20 Sep 2017 at 20:41 Kunal Khatua wro

RE: User client timeout with results > 2M rows

a:357) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadE > ventExecutor.java:111) [netty-common-4.0

RE: Querying Parquet files in 2.0 format

Drill currently is supporting Parquet 1.8.1 https://github.com/apache/drill/blob/master/pom.xml#L37 AFAIK a lot of major projects are still on Parquet 1.x versions. Is there something specific in 2.0 that you need? Else, Parquet 1.8.1 should suffice. You needn’t downgrade all the way down to 1.0

RE: Error Messages that are difficult to parse.

Not sure what is going on, but my hunch is that the outermost wrapping SQL is probably using the final projections to eliminate some of the columns early on, which "helps" avoid the NumberFormat exception. Perhaps adding back the other columns, one by one, should narrow down the source of the

Re: need help to decrypt error code

This appears to be the brief client side error. What does the Drillbit 's log show in the stacktrace? From: Divya Gehlot Sent: Thursday, September 28, 2017 12:43:38 AM To: user@drill.apache.org Subject: need help to decrypt error code Hi, I have 18 MB csv file a

RE: Access to Drill 1.9.0

Just curious... any reason why you're looking to try Drill 1.9.0, considering that is nearly a year old ? -Original Message- From: Rob Wu [mailto:robw...@gmail.com] Sent: Friday, October 06, 2017 10:35 PM To: user@drill.apache.org Subject: Re: Access to Drill 1.9.0 Hi Chetan, You can

RE: Exception while reading parquet data

If this resolves the issue, could you share some additional details, such as the metadata of the Parquet files, the OS, etc.? Details describing the setup is also very helpful in identifying what could be the cause of the error. We had observed some similar DATA_READ errors in the early iterati

RE: Exception while reading parquet data

am.read(BufferedDirectBufInputStream.java: > > 277) ~[drill-java-exec-1.11.0.jar:1.11.0] > > at org.apache.drill.exec.util.filereader. > > DirectBufInputStream.getNext(DirectBufInputStream.java:111) > > ~[drill-java-exec-1.11.0.jar:1.11.0] > > at org.apache.d

  1   2   3   >