I have some JSON that includes a field that is a array type, and some of
those fields are json nulls.
As seen in my other recent post about views and the "Please set
`store.json.all_text_mode` I am getting this error here.
Like I put in my other post, I don't want to set this at a system level,
nor do I want to set it at a session level, instead, I'd only want to set
it when querying this directory of json... so, ignoring that (if you want
to talk about that head to my other post on the subject :)
In the mean time, I thought I could just exclude the field with a view
When I select all the fields except the one that has the error, I still get
the error. That tells me Drill is reading the entire line (obviously it has
to do this) it gets the keys it's going to be parsing, and then reads and
attempts to parse the values prior to it knowing if it will need them.
Could this be a point of optimization and error catching?
Some silly example data:
{"field1":"value1", "field2":"value2", "field3":[null, "some1", "some2"]}
if I do
select * from `data` I get the error due to field3's value (null)
if I do select count(field1) from `table
That works.
So if I make a view:
CREATE myview as
select field1, field2, dir0 from `table`
Now select count(field1) from myview works
but select count(*) from myview does not work...
Is there am optimization that is happening somewhere that is forcing a
table read? That field should be excluded... in addition
select count(dir0) from myview and select count(dir0) from `table` both
fail as well... which is interesting to me, it's like that field (field3)
is trying to be processed when it shoudn't have to be based on what we are
querying...
I am a bit scattered on this (in my explanation) if anyone would like more
examples or more explanation on this let me know.
Basically I am trying to figure out a way to get around this error without
forcing a user to change the session (or the system) variable.
Thanks!
John
