Re: empty results

2015-07-16 Thread Stefán Baxter
both are empty sets (with headers) On Jul 16, 2015 5:31 PM, Sudheesh Katkam skat...@maprtech.com wrote: Is it returning empty results only though REST API? Did you try sqlline? Do you have a simple repro? If so, can you file a ticket? Thank you, Sudheesh On Jul 16, 2015, at 10:25 AM,

Re: Drill with S3 without hardcoding credentials into core-site

2015-07-16 Thread Paul Mogren
Hafiz, I have not worked with this at all, but wanted to make sure you saw https://drill.apache.org/docs/configuring-user-impersonation/ and https://drill.apache.org/docs/configuring-user-authentication/ Maybe there is an implementation of PAM that integrates with IAM... -Paul On 7/13/15,

Re: How to configure HDFS storage

2015-07-16 Thread Arthur Chan
On Wed, Jun 17, 2015 at 10:47 PM, Arthur Chan arthur.hk.c...@gmail.com wrote: Hi, Thanks! There is one thing about connection that I feel confused: In the HA Hadoop Cluster, instead of use ip_address and port, we use zookeeper QuorumPeerMain and use dfs.nameservices to manage the

Re: Operation category READ is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.Standby

2015-07-16 Thread Arthur Chan
Anyone has idea what I would be wrong in setup Drill? On Tue, Jul 14, 2015 at 4:21 PM, Arthur Chan arthur.hk.c...@gmail.com wrote: Hi, I have HDFS HA with two namenodes (nn1 and nn2 respectively) When the namenode nn1 is failover to nn2, when querying HIVE, I got the following error:

Querying Apache Spark Generated Parquet

2015-07-16 Thread Usman Ali
Hi, I am having trouble querying a parquet file generated using Apache Spark. *Select * from `file.parquet` * works fine but when I try to select only some fields of parquet files it returns Null values. i.e.* Select field1 from `file.parquet` * returns only Null values. No, field1 does not

using the REST API

2015-07-16 Thread Stefán Baxter
Hi, I have a few questions regarding the rest API. - Is it possible that the rest api (query.json) should return numeric values as strings? - count(*) being an example of that - calls for conversion on the browser side - I find no obvious setting for this - Is there any other

Re: using the REST API

2015-07-16 Thread Sudheesh Katkam
See inline. On Jul 16, 2015, at 4:36 AM, Stefán Baxter ste...@activitystream.com wrote: Hi, I have a few questions regarding the rest API. - Is it possible that the rest api (query.json) should return numeric values as strings? There is a ticket for this:

Re: using the REST API

2015-07-16 Thread Stefán Baxter
thanks Sudheesh, it's appreciated. On Thu, Jul 16, 2015 at 4:58 PM, Sudheesh Katkam skat...@maprtech.com wrote: See inline. On Jul 16, 2015, at 4:36 AM, Stefán Baxter ste...@activitystream.com wrote: Hi, I have a few questions regarding the rest API. - Is it possible that the

Re: Rest API

2015-07-16 Thread Sudheesh Katkam
See inline. On Jul 16, 2015, at 4:03 AM, Preetham Nadig preetham.na...@ariplaso.com wrote: Hello, I am planning to use DRILL for one of my projects and I have been working with it for couple of weeks That’s awesome! One of the things I would like to do is access DRILL over a REST API,

empty results

2015-07-16 Thread Stefán Baxter
Hi, What can be happening if a drillbit (local) starts returning empty results (from a Parquet query) and does not return proper results unless it's restarted? (I noticed this started happening when I began using the REST API but I have no direct link to that) Regards, -Stefán

Re: Flatten Output Json

2015-07-16 Thread Sudheesh Katkam
Does this help? http://drill.apache.org/docs/flatten/ http://drill.apache.org/docs/flatten/ Thank you, Sudheesh On Jul 15, 2015, at 10:55 PM, Usman Ali usman@platalytics.com wrote: Hi, Drill sqlline displays output in a nice format. I am guessing it must be flattening the output

Re: Drill on Mesos - A Story

2015-07-16 Thread Timothy Chen
Also will be nice to launch Drill with a docker image so no tar ball is needed, and much easier be cached on each slave. Tim On Jul 16, 2015, at 9:37 AM, John Omernik j...@omernik.com wrote: Awesome thanks for the update on memory! On Thu, Jul 16, 2015 at 10:48 AM, Andries Engelbrecht

Hive Metastore

2015-07-16 Thread Ascot Moss
Hi Drill team, On the Drill documentation page, https://drill.apache.org/docs/file-system-storage-plugin/, it states that the IP address is required if HDFS is used. • HDFS connection: hdfs://IP Address:Port/ e.g. connection: hdfs://10.10.30.156:8020/ My question is that: Does the

Re: Set Drill Response Format to CSV Through Rest APIs

2015-07-16 Thread Sudheesh Katkam
Currently we support only JSON through REST API. Thank you, Sudheesh On Jul 15, 2015, at 9:26 PM, Usman Ali usman@platalytics.com wrote: Hi, Is there any way to set response format of drill to csv instead of json using Rest APIs? If yes, then what other response formats are

Re: Operation category READ is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.Standby

2015-07-16 Thread Sudheesh Katkam
Can you try just “thrift://nn2:9083 thrift://nn2:9083” (and not include the failover namenode) for “hive.metastore.uris” property? Thank you, Sudheesh On Jul 16, 2015, at 1:43 AM, Arthur Chan arthur.hk.c...@gmail.com wrote: Anyone has idea what I would be wrong in setup Drill? On Tue,

Re: Drill on Mesos - A Story

2015-07-16 Thread Timothy Chen
Hi John, Fetcher cache is going to be in 0.23 so that's something that you can leverage. You'll find more doc in the docs folder about it once we have 0.23 released. Tim On Thu, Jul 16, 2015 at 12:08 PM, John Omernik j...@omernik.com wrote: Timothy - I played with that, and the performance

Re: Drill on Mesos - A Story

2015-07-16 Thread John Omernik
Timothy - I played with that, and the performance I was getting in Docker was about half that I was getting native. I think that for me, that was occurring because if I ran it in Docker, I needed to install the MapR Client in the container too, whereas when I run it in marathon, it's using the

Re: Recursive CTE Support in Drill

2015-07-16 Thread Ted Dunning
Also, just doing a Cartesian join of three copies of 1000 records will give you a billion records with negligible I/o. Sent from my iPhone On Jul 16, 2015, at 15:43, Jason Altekruse altekruseja...@gmail.com wrote: @Alexander If you want to test the speed of the ODBC driver you can do that

Re: Operation category READ is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.Standby

2015-07-16 Thread Venki Korukanti
Hi Arthur, Property hive.metastore.uris should contain the URI of the remote metastore. For example: thrift://hivemetahost:9083, here Hive metastore is running on host hivemetahost. Configuration properties related to NameNode HA should be part of the configProps section. These properties should

Re: Recursive CTE Support in Drill

2015-07-16 Thread Alexander Zarei
Thanks for the answers. @Ted my only goal is to pump a large amount of data without having to read from Hard Disk. I am measuring the ODBC driver performance and I need a higher data transfer rate. So any method that helps pumping data out of Drill faster would help. The log-synth seems a good

Re: Recursive CTE Support in Drill

2015-07-16 Thread Jason Altekruse
@Alexander If you want to test the speed of the ODBC driver you can do that without a new storage plugin. If you get the entire dataset into memory, it will be returned from Drill a quickly as we can possibly send it to the client. One way to do this is to insert a sort; we cannot send along any

Re: How to configure HDFS storage

2015-07-16 Thread Venki Korukanti
Hi Arthur, Currently schema of FileSystem storage plugin doesn't allow you to any configuration property other than fs.default.name (basically the connection string) for configuration. It would be good if we have config section similar to hive or hbase storage plugins. Please log a jira for

Re: Rrounding timestamps to nearest period interval

2015-07-16 Thread Stefán Baxter
Thank you Christopher I will build my workaround on this. -Stefan On Wed, Jul 15, 2015 at 8:49 PM, Christopher Matta cma...@mapr.com wrote: I did this last week with a query on the Drill profiles, this query will break down the number of queries in 5 minute increments: select x.`timestamp`,