Error parsing JSON

2016-10-12 Thread Dan Blondowski
Hello. I have a json file with 2 records (see attached). When I run a query against it I get this error: Error: DATA_READ ERROR: Error parsing JSON - You tried to write a BigInt type when you are using a ValueWriter of type NullableVarCharWriterImpl. File /test.json Record 2 Fragment 0:0

Re: Error parsing JSON

2016-10-12 Thread Khurram Faraaz
Can you please share your SQL query ? On Wed, Oct 12, 2016 at 7:38 PM, Dan Blondowski < dan.blondow...@dhigroupinc.com> wrote: > Hello. > > I have a json file with 2 records (see attached). > > When I run a query against it I get this error: > > *Error: DATA_READ ERROR: Error parsing JSON - You t

Re: Error parsing JSON

2016-10-12 Thread Dan Blondowski
Sorry! Select * from s3.`test.json`; Daniel Blondowski Big Data Architecture/Engineering DHI Group, Inc. dan.blondow...@dice.com 515-313-2137 On 10/12/16, 9:52 AM, "Khurram Faraaz" wrote: >Can you please share your SQL query ? > >On Wed, Oct 12, 2016 at 7:38 PM, Dan Blondowski < >dan.bl

Re: Error parsing JSON

2016-10-12 Thread Khurram Faraaz
Can you please try to set this config parameter from sqlline and then run your query. alter session set `store.json.all_text_mode` = true; On Wed, Oct 12, 2016 at 8:38 PM, Dan Blondowski < dan.blondow...@dhigroupinc.com> wrote: > Sorry! > Select * from s3.`test.json`; > > Daniel Blondowski > >

Re: Error parsing JSON

2016-10-12 Thread Abhishek Girish
Hey Dan, This usually happens when there is a schema change across records. It could be intentional (records do need different types for some fields) or bad formatting (1 vs "1"). Can you try setting this session option and retry your query? Let us know if it helps. You could attempt to use expli

Re: Error parsing JSON

2016-10-12 Thread Dan Blondowski
Hello. You were correct. I was able to set `store.json.all_text_mode` = true; and run the query. I also found the data that needed changing. There were actually 3 fields. Is there a way for Drill to display which field is bad, so I don¹t have to do the trial & error? On 10/12/16, 10:12 AM,

drill distributed mode on Windows: No DrillbitEndpoint can be found

2016-10-12 Thread Anton Kravchenko
Hi there, I wonder if Apache Drill is supported in distributed mode on Windows? If so, what is an alternative for 'drillbit.sh start' command? P.S. I've successfully tested Drill in embedded mode on two Windows boxes, and in distributed mode on four Linux boxes. Thank you, Anton

Re: PAM Authentication in Embedded Mode

2016-10-12 Thread Muthu pandi
Hello Chun Chang Thanks for reply. I want to know where to provide username and password.? If i fire this bin/drill-embedded -n username -p password, then what will be my username and password.? where to set these.?

Re: PAM Authentication in Embedded Mode

2016-10-12 Thread Chun Chang
That's one of your Linux user and password on that machine. On Wed, Oct 12, 2016 at 12:27 AM, Muthu pandi wrote: > Hello Chun Chang > Thanks for reply. I want to know where to provide username > and password.? > > If i fire this bin/drill-embedded -n username -p password, > then

Re: Error parsing JSON

2016-10-12 Thread Khurram Faraaz
Since your JSON file has different types of data (line int, boolean, strings etc) we need to tell Drill to consider all data in that JSON file as text. This is by design, and this is not trial and error. Such occurrence of different types of data in a JSON file is called SchemaChange and to avoid t

Re: Error parsing JSON

2016-10-12 Thread Dan Blondowski
Yes, I understand that part. I wasn’t clear enough. By trial and error, I meant - me trying to find which field was bad. I’m going to ask the guys that provide the source file, to make sure data types for each field are consistent across records. On 10/12/16, 12:53 PM, "Khurram Faraaz" wro

Re: Error parsing JSON

2016-10-12 Thread Andries Engelbrecht
Look into the JSON data model http://drill.apache.org/docs/json-data-model/ You can set the union type to true for the session and the use the typeof function to determine data types of the fields. Perhaps do a group by typeof. exec.enable_union_ty

Re: drill distributed mode on Windows: No DrillbitEndpoint can be found

2016-10-12 Thread Chun Chang
It may not work out of the box but in theory it should work. May need to model sqlline.bat file and create a windows version of drillbit.sh. On Wed, Oct 12, 2016 at 10:17 AM, Anton Kravchenko < kravchenko.anto...@gmail.com> wrote: > Hi there, > > I wonder if Apache Drill is supported in distribut

Re: Error parsing JSON

2016-10-12 Thread Abhishek Girish
Unfortunately, the only information shared is the file name and the record having the issue. In your example, you can see that the issue was with record 2 in file /test.json. You will need to use trial and error method to isolate the field in my opinion. On Wed, Oct 12, 2016 at 9:01 AM, Dan Blondo

Drill connecting to Azure DocumentDB store

2016-10-12 Thread Sameer Nori
Can Drill connect to the Azure Document DB store? Thanks, -- Sameer Nori Sr Product Marketing Manager +1.7037250917 @sameernori Now Available - Free Hadoop On-Demand Training [image: http:/

Setting up HDFS access on EMR

2016-10-12 Thread David Kincaid
I have an Amazon EMR cluster launched with Drill loaded. I'm trying to configure the dfs storage plugin to use HDFS. The docs say that I should simply need to change the "connection" setting from "file:///" to "hdfs://" in order to use HDFS on the cluster that Drill is running on. However, when I d

Re: Setting up HDFS access on EMR

2016-10-12 Thread Ted Dunning
What is the full URL you used? With hdfs://, you need to supply a name node address. With file://, you don't. Contrarily, with maprfs:// you don't need an address since it is implied in the client connection. On Wed, Oct 12, 2016 at 6:29 PM, David Kincaid wrote: > I have an Amazon EMR clust