Hi Dan, Thanks for bringing up this question. Such behavior was intentional since Avro files already have schema and Drill is able to use it during queries validation. It was done in the scope of DRILL-3810 <https://issues.apache.org/jira/browse/DRILL-3810>. We have similar behavior for Hive tables - to fail if a non-existent column is queried.
But I think it would be good to optionally allow returning null for Avro tables to be consistent with other file storage formats. Could you please log a Jira for this improvement? Kind regards, Volodymyr Vysotskyi On Tue, Oct 8, 2019 at 10:14 PM Dan Schmitt <dan.schm...@nielsen.com> wrote: > Similar to the "Check presence of field in json file" of last week, I was > hoping to be able to fire drill off at avro files with evolving and > variant schemas. > (For the case where a field gets added over time, or a field is optional.) > > One of the suggestions for the json version was > > sqlTypeOf(someField) > > to help filter out files that don't have that value. The avro parser > chokes > with > > Error: VALIDATION ERROR: From line 1, column 18 to line 1, column 21: > Column 'someField' not found in any table > > instead of returning null (and the suggested EXISTS/ IS NOT NULL where > clauses > are also failing.) > > Is there a JIRA issue to support where clauses to sift out missing > fields for Avro files? > > Dan S. >
