Hi Drill users,
I'm having an issue with an academic dataset that I am attempting to process
with Drill 1.6.0. The dataset contains a field which can either be represented
as a double, or as a boolean, which is causing queries to fail.
To reproduce this error, use this query:
select * from hdfs.`/tmp/test.json`;
On this dataset:
{"modified":false}
{"modified":1423015689}
Which should produce this error:
DATA_READ ERROR: Error parsing JSON - You tried to write a BigInt type when you
are using a ValueWriter of type NullableBitWriterImpl.
If there is not a Drill configuration to resolve this issue? I understand that
the option store.json.all_text_mode will read in all values as text which will
resolve the initial error, however I am unsure of what the drawbacks of
removing strong type checking will be. I would also be fine pre-processing the
data to convert the false boolean to null, which I believe should solve the
issue. Do you have recommendations on how to best perform either of these
actions? I appreciate any help you can give me on this.
Thanks,
Ben
