Re: Schema problems trying to convert JSON to Parquet

2018-02-23 Thread Paul Rogers
Hi David, Your b.json file has only nulls; there is no way for Drill to determine what type of null is in your file. Drill requires each NULL to be a null of some type. Often, Drill guesses nullable int, which is why you saw the problem in your query. If all your fields are strings, there is a

RE: Schema problems trying to convert JSON to Parquet

2018-02-23 Thread Lee, David
user@drill.apache.org Subject: Re: Schema problems trying to convert JSON to Parquet This is a challenge when dealing with JSON. You can either force the data type in the CTAS statement (likely better option) or deal with the data type change in parquet table(s) by using CAST, etc. In the case o

RE: Schema problems trying to convert JSON to Parquet

2018-02-23 Thread Lee, David
user@drill.apache.org Subject: Re: Schema problems trying to convert JSON to Parquet This is a challenge when dealing with JSON. You can either force the data type in the CTAS statement (likely better option) or deal with the data type change in parquet table(s) by using CAST, etc. In the case of zip code

Re: Schema problems trying to convert JSON to Parquet

2018-02-23 Thread Andries Engelbrecht
This is a challenge when dealing with JSON. You can either force the data type in the CTAS statement (likely better option) or deal with the data type change in parquet table(s) by using CAST, etc. In the case of zip codes you need to consider if it will be 5 digits or the extended 5-4 digits to