Re: query plan caching?

2016-01-14 Thread Jason Altekruse
Currently not to my knowledge. Are there queries you are seeing that are taking an abnormally long time to plan? On Thu, Jan 14, 2016 at 6:25 AM, Vince Gonzalez wrote: > Does Drill do any caching of query plans? >

Re: directory create CTAS behaviour

2016-01-14 Thread Leon Clayton
I don’t require the output to go to a single file i just want to remove the sub folder creation on the output location.. Regards Leon Clayton Solutions Architect | +44 (0)7530 980566 MapR Technologies UK Ltd. Now Available - Free Hadoop On-Demand Training > On

Re: query plan caching?

2016-01-14 Thread Andries Engelbrecht
Most of the effort have been going into metadata caching and improvements in this area, specifically DFS parquet and Hive. And also improved directory pruning at planning time. This helps to reduce he planning time significantly. These seem to have been the big time consumers for query plans I

Re: JSON File, Total numbers Record: 1

2016-01-14 Thread Jason Altekruse
You can use * to propagate all traits up. The only problem with this is that it will pull up a copy of the list you are flattening as well (copying the complete list next to each record in the flattened out version of the list). Databases have always frustratingly lacked a feature of 'select *

Re: 1.5-SNAPSHOT and UDFs

2016-01-14 Thread Stefán Baxter
Hi Jacques, I'm still struggling with this. The function you seem to be pointing me to is protected and I can not call it directly from my code. I created the same function in a local utility class: package com.activitystream.drill; public class ASDrillUtils { public static String

Re: 1.5-SNAPSHOT and UDFs

2016-01-14 Thread Stefán Baxter
Hi again, I modified this slightly and now it works (it's accepting ValueHolder and resolving that function correctly) regards, -Stefan On Thu, Jan 14, 2016 at 8:25 AM, Stefán Baxter wrote: > Hi Jacques, > > I'm still struggling with this. The function you seem to

RE: Querying a directory of json.gz Files in Drill 1.4

2016-01-14 Thread Holy Alexander
Dear Kristine, Still fighting with 1.4 I carefully checked for hidden and system files - there are none Also keep in mind, that the very same query and same data directories do work fine in 1.3 (on the same machine, with the same setup) From a user point of view it seems that directory

query plan caching?

2016-01-14 Thread Vince Gonzalez
Does Drill do any caching of query plans?

Re: directory create CTAS behaviour

2016-01-14 Thread Neeraja Rentachintala
What would you like to see instead of directory. On Thursday, January 14, 2016, Leon Clayton wrote: > Hello All > > Is it possible to change this behaviour. By default, a directory is > created, using the exact table name specified in the CTAS statement. I > don’t want a

RE: JDBC Ignores URL ?

2016-01-14 Thread Timothy Findlay
Hi Folks, Thanks for the feedback, I kept playing with the URL's and ended up with: jdbc:drill:drillbit=192.168.56.102:31010 Using that I think I might have some insight into the core problem. I omitted from the initial report was that I had dropped an Oracle JDBC JAR in and tried to

Lucene Plugin :: Join Filter and pushdown

2016-01-14 Thread Stefán Baxter
Hi, I'm working on the Lucene plugin (see previous email) and the focus now is support for joins with filter push-down to avoid the default table scan that is provided by default. I'm fairly new to Drill and in over my head, to be honest, but this is fun and with this addition the Lucene plugin

Re: "Already had POJO for id" Error In SQL - Join B/W Redshift and HDFS

2016-01-14 Thread Jason Altekruse
Jacques, not sure if you caught this, in the stacktrace it mentions broadcast sender. Did the plan for your test query include a broadcast join? * (com.fasterxml.jackson.databind.JsonMappingException) Already had POJO for id (java.lang.Integer)

Fwd: Drill query does not return all results from HBase

2016-01-14 Thread Jason Altekruse
Thanks for the update, I'm forwarding your message back to the list. Just to confirm, was the query time longer on the the one with the incorrect result? In the incorrect case I think we are just misreading the HBase metadata during our optimization to return row counts without reading any data.

Re: query plan caching?

2016-01-14 Thread Neeraja Rentachintala
Query plan caching is typically very useful if you have reporting type of queries where the query patterns are fixed and vary mostly on filter conditions. For adhoc queries or data exploration use cases where there may not be fixed query patterns, this technique is not frequently useful. Similar

Re: "Already had POJO for id" Error In SQL - Join B/W Redshift and HDFS

2016-01-14 Thread Jacques Nadeau
Hey Rohit, I'm having trouble reproducing this in my environment (pointing at derby + hdfs instead of redshift/postgres). Can you turn on debug logging and then run this query? You can enable the debug logging we are interested in by adding the following item to logback.xml: