Re: Convert UTC to specific timezone?

2015-03-31 Thread Ted Dunning
Ah... if you have datasets with other time zones, that is relatively easily done. Just convert all of the times into time stamps and compare to your hearts content. If your data has an implied time-stamp, you can append a correct one and use the z format specifier to parse the concatenated string

Re: mongodb authentication problem

2015-03-31 Thread Wang Darrell
Thanks for your answer , I tried both two links and still having the same error , CommandFailureException: { "serverUsed" : "192.168.170.136:27017" , "ok" : 0.0 , "errmsg" : "not authorized on admin to execute command { listDatabases: 1 }" my mongodb storage config { "type": "mongo", "connec

[ANNOUNCE] Release of Apache Drill 0.8.0

2015-03-31 Thread Jacques Nadeau
Hello Everyone, I'm pleased to announce the release of Apache Drill 0.8.0. There are a number of exciting additions in this release and a huge number of bug fixes. Read about the release in our release highlights at [1]. Please download it [2] and let us know what you find. If you want to view

Convert UTC to specific timezone?

2015-03-31 Thread Vince Gonzalez
On Tuesday, March 31, 2015, Andries Engelbrecht wrote: > TMZ functionality will be good, especially if it can be specified in a > query and also set at session level. > > Another question is if Drill will assume all date/time data is UTC unless > specified differently? I believe this to be the ca

Re: Hive SerDe Error

2015-03-31 Thread David Tucker
I believe you’ll need to put the custom jar in DRILL’s classpath (it does not include $HIVE_HOME/lib be default since there’s no guarantee it will be on all cluster nodes). I’ve been successful putting the extra libraries I need for object-store access into $DRILL_HOME/jars/3rdparty. — Davi

Re: WHERE clause with nested JSON data

2015-03-31 Thread Jason Altekruse
The error message indicates that this is a planning bug. Please try to look to see if you can find an open JIRA for the issue and add any information about your case there. If there is not one already filed, please open a new one and try to provide as much explanation as you can about the data invo

Hive SerDe Error

2015-03-31 Thread Ganesha Muthuraman
Posting this message again as I did not get a response to the previous: I am getting a hive custom SerDe not found error when I try to describe (or query) hive tables in drill that use a custom SerDe. show tables works ok. Also, any table that does not use custom SerDe, works well. All tables work

Re: WHERE clause with nested JSON data

2015-03-31 Thread Kristine Hahn
These examples of nested data queries that use a where clause might help: http://drill.apache.org/docs/json-data-model/#example:-access-a-map-field-in-an-array http://drill.apache.org/docs/json-data-model/#example:-flatten-an-array-of-maps-using-a-subquery Kristine Hahn Sr. Technical Writer 415-

Re: WHERE clause with nested JSON data

2015-03-31 Thread Andries Engelbrecht
Try a cast to integer on the field as well, just to be sure data types are handled correctly. —Andries On Mar 31, 2015, at 8:54 AM, Subhajit Ghosh wrote: > Checked it, there aren't any null values. Any queries with the WHERE clause > containing a nested statement like "WHERE a.b.c>100" is pr

Re: WHERE clause with nested JSON data

2015-03-31 Thread Subhajit Ghosh
Checked it, there aren't any null values. Any queries with the WHERE clause containing a nested statement like "WHERE a.b.c>100" is producing the same error. Anyone succeeded with such queries? On 31 March 2015 at 16:27, Andries Engelbrecht wrote: > Are there any null values? > > Perhaps try to

Re: Question on Drill Distributed Mode

2015-03-31 Thread Andries Engelbrecht
I would suggest that you use a tool to split the JSON file into smaller chunks of 64-128MB, and keeping the JSON objects intact in each file. Also it will be best in the long run to just use a distributed FS for the Drill cluster vs trying to manage file partitions. I personally prefer MapR-FS as

Re: WHERE clause with nested JSON data

2015-03-31 Thread Andries Engelbrecht
Are there any null values? Perhaps try to filter out null values (if there are any) and then see if it succeeds. Select <….> from …. t where t.json.runtimeConfiguration.properties.jvmHeapUsageInit is not null and t.json.runtimeConfiguration.properties.jvmHeapUsageInit >3000 In some cases I fo

Re: mongodb authentication problem

2015-03-31 Thread Andries Engelbrecht
See the following links. http://drill.apache.org/blog/2014/11/19/sql-on-mongodb/ https://issues.apache.org/jira/browse/DRILL-1502 —Andries On Mar 31, 2015, at 2:24 AM, Wang Darrell wrote: > Is there anyone who tried to connect drill to mongodb with --auth. > > When I started my mongo with a

Re: Convert UTC to specific timezone?

2015-03-31 Thread Andries Engelbrecht
TMZ functionality will be good, especially if it can be specified in a query and also set at session level. Another question is if Drill will assume all date/time data is UTC unless specified differently? I believe this to be the case. Date_add, Date_sub seems to only support days. You can als

Re: Convert UTC to specific timezone?

2015-03-31 Thread Christopher Matta
Ted's correct, it would be nice to be able to convert the UTC datetime column to whichever timezone I'm interested in, say 'America/New_York', as a timestamp so I can compare datasets that don't have UTC timestamps with those that do. Is this in the roadmap? Chris Matta cma...@mapr.com 215-701-314

mongodb authentication problem

2015-03-31 Thread Wang Darrell
Is there anyone who tried to connect drill to mongodb with --auth. When I started my mongo with auth. drill can't do any query. My mongodb version is 2.6.9 here is the mongodb admin user information db.createUser({ user:"test", pwd :"test", roles:[{ role : "readAnyDatabase" , db:"admin" }] })

WHERE clause with nested JSON data

2015-03-31 Thread Subhajit Ghosh
I am facing some issues when running a SELECT query with a WHERE clause on a nested value/column. Note that the query is run on a view of the HBase table. 0: jdbc:drill:schema:hbase:zk=localhost> select t.json.runtimeConfiguration.properties.jvmHeapUsageInit as val from IndividualTestRun_ t; +

Re: Convert UTC to specific timezone?

2015-03-31 Thread Ted Dunning
The original poster wasn't very clear. What they said could mean what Andries provided (which is to determine which timezone that data refers to). The way that I read the question was that they wanted to translate times to be represented as the string formatted version of the same time in a diffe

Question on Drill Distributed Mode

2015-03-31 Thread Varun Kumar Reddy B
Hello Team I started exploring drill for our requirement to run SQL-on-semi structured data. I have setup a 4node drill cluster with zookeeper. Have few questions on how it actually works, 1. When I run Drill in distributed mode, using dfs (local file system) i.e., I have a 1GB Json file on one