Re: directory pruning and UDFs

Can you also provide some examples of what you are trying to accomplish? It seems like you might be saying that you want a virtual attribute for the entire path rather than individual pieces? Also remember that partition pruning can also be done if you're using Parquet files without all the dirN

Re: null values while querying json files

Hi Andries, I've already tried writing where and case statements but none of them worked unfortunately. It's still: DATA_READ ERROR: Error parsing JSON - You tried to write a VarChar type when you are using a ValueWriter of type NullableIntWriterImpl. When I try this on sqlline, I can confirm

Re: NullPointers in type conversions

Does this system option not work: ALTER SESSION SET `drill.exec.functions.cast_empty_string_to_null` = true; The reason the bug was marked INVALID is that SQL engines (not sure about the spec) don't allow casting from empty string to number. The system option above is supposed to allow changing

Re: null values while querying json files

This is a limitation of really late appearing fields. Right now, depending on the situation, if a value doesn't show up in the first ~4k, we assume that the value is BigInt. I think a developer is working on improving this behavior right now. I'll ping them to see when we might have a fix. --

Re: null values while querying json files

thank you, it's greatly appreciated. On Fri, Sep 18, 2015 at 10:24 PM, Julien Le Dem wrote: > I'm going to look into this next week. > Here is a JIRA to follow the issue, comment, suggest solutions: > https://issues.apache.org/jira/browse/DRILL-3806 > > On Fri, Sep 18, 2015

Re: null values while querying json files

I'm going to look into this next week. Here is a JIRA to follow the issue, comment, suggest solutions: https://issues.apache.org/jira/browse/DRILL-3806 On Fri, Sep 18, 2015 at 3:10 PM, Stefán Baxter wrote: > Hi, > > I have nothing meaningful to add but I had share

Re: null values while querying json files

Hi, I have nothing meaningful to add but I had share that this BigInt assumption has caused more grief than any other single feature in Drill. I would go so far as to say that the "type intolerance" and lack of "sensible conversion" is the biggest hurdle on the way to fulfilling the "eliminate

CTAS exception

Hi, I have some json files that I want to transform to parquet. We have been doing this without any issues but this time around I get this exception: Error: SYSTEM ERROR: IllegalStateException: Failure while reading vector. Expected vector class of

Re: CTAS exception

This kind of errors usually happens when there is an unsupported schema change in the json files, but you should be able to reproduce the error with just a select statement. Can you share both queries you tried (the failing CTAS and the successful SELECT *) ? Thanks On Fri, Sep 18, 2015 at 5:38

Re: CTAS exception

Hi, Both statements select everything but the CTAS statement included a date filter + date order. The date field is always the same (extended json format date ISO) and is always present so I can say, with 100% certainty, that there is no schema change involved. The select statement fails as

Re: CTAS exception

Hi, There are no schema changes. I can select * from the table just fine! Regards, -Stefán On Fri, Sep 18, 2015 at 4:55 PM, Abhishek Girish wrote: > Are you sure there is no schema change occurring between records for a > column other than *occurred_at?**. *In

Re: CTAS exception

This could be a bug. Please open a new Jira and add as much information as possible ? thanks On Fri, Sep 18, 2015 at 10:07 AM, Stefán Baxter wrote: > Hi, > > There are no schema changes. > > I can select * from the table just fine! > > Regards, > -Stefán > > On Fri,