Avro storage format behaviour

[Vova Vysotskyi]

Hi all,

I am working on DRILL-4120: dir0 does not work when the directory structure
contains Avro files.

In DRILL-3810 was added validation of query using avro schema before start
executing the query.
Therefore with these changes Drill throws an exception when the
query contains non-existent column and table has avro format.
Other storage formats such as json or parquet allow usage of non-existing
fields.

So here is my question: should we continue to treat avro as a format with
fixed schema, or we should start treating avro as a dynamic format to be
consistent with other storage formats?

-- 
Kind regards,
Volodymyr Vysotskyi

Re: Base64 encoded results for Drill query (>10.000 columns)

[Arina Yelchiyeva]

Hi Daniel,

not sure if there is another way. Looks like you'll have to use CONVERT_FROM
function for each column.

Kind regards
Arina

On Fri, Feb 23, 2018 at 10:56 AM, Daniel Müller <
daniel.muel...@hs-offenburg.de> wrote:

> Hi there,
>
> I started working with Drill a few weeks ago and I'm still wondering why
> the query results are Base64 encoded...
>
> I found this ticket, which also handles this situation:
> https://issues.apache.org/jira/browse/DRILL-4620
>
> Of course, there's the CONVERT_FROM function to translate the results into
> the readable result, but as I'm working on a very dynamic HBase table
> (>10.000 columns, new columns are added every day), I can't call this
> convert function on each of these columns.
>
> Is there another way to convert all returned columns automatically? Thank
> you!
>
> Best regards
> Daniel Müller
>
>