Similar to the "Check presence of field in json file" of last week, I was
hoping to be able to fire drill off at avro files with evolving and
variant schemas.
(For the case where a field gets added over time, or a field is optional.)
One of the suggestions for the json version was
sqlTypeOf(someField)
to help filter out files that don't have that value. The avro parser chokes
with
Error: VALIDATION ERROR: From line 1, column 18 to line 1, column 21:
Column 'someField' not found in any table
instead of returning null (and the suggested EXISTS/ IS NOT NULL where clauses
are also failing.)
Is there a JIRA issue to support where clauses to sift out missing
fields for Avro files?
Dan S.
