Hi,
This turned out to be a incomplete union:
- {"name": "market", "type": ["string"]}
Avro tools ignored it and created the classes as if this was a normal
string field.
I would really appreciate it if the error messages Drill produces would be
structured in a more helpful way.
Knowing what went wrong is fine, knowing why it went wrong is even better
and I think that in this case it should be easy to include a piece of the
header that is being parse/processed.
Regards,
-Stefan
On Fri, Oct 16, 2015 at 4:45 PM, Stefán Baxter <[email protected]>
wrote:
> Hi Kamesh,
>
> I pulled, made and deployed 1.2-SNAPSHOT (master/head/"Latest commit
> 12fb9a
> <https://github.com/apache/drill/commit/12fb9ac11e25a2bbd9fff5d9cdb6a355b96ffadb>
> 3 days ago") producing the same results.
>
> I can not see a RC3 Branch, am I missing something?
>
> Regards,
> -Stefan
>
>
>
>
> On Fri, Oct 16, 2015 at 4:31 PM, Stefán Baxter <[email protected]>
> wrote:
>
>> Hi Kamesh,
>>
>> I must be lagging behind then, let me double check.
>>
>> Regards,
>> -Stefan
>>
>> On Fri, Oct 16, 2015 at 4:23 PM, Kamesh <[email protected]> wrote:
>>
>>> Hi Stefan,
>>> Do you have latest code changes with you (1.2-SNAPSHOT)? Could you
>>> please
>>> try RC3 build.
>>> Yes, it supports nested records.
>>>
>>>
>>> On Fri, Oct 16, 2015 at 8:28 PM, Stefán Baxter <
>>> [email protected]>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > Sorry I forgot to mention that I'm already using a 1.2-SNAPSHOT
>>> >
>>> > Should it be supporting Records as a data type? (use to map sub
>>> > structures).
>>> >
>>> > Regards,
>>> > -Stefán
>>> >
>>> >
>>> >
>>> >
>>> > On Fri, Oct 16, 2015 at 2:31 PM, Kamesh <[email protected]>
>>> wrote:
>>> >
>>> > > Hi Stefan,
>>> > > We have enhanced Drill Avro Reader by supporting complex data types
>>> like
>>> > > Union, Map etc.. This is targeted for 1.2.0 release. This release
>>> will
>>> > > happen in a day or in a couple days.
>>> > >
>>> > > On Fri, Oct 16, 2015 at 6:30 PM, Stefán Baxter <
>>> > [email protected]>
>>> > > wrote:
>>> > >
>>> > > > Hi,
>>> > > >
>>> > > > I'm changing our logging to use the Avro format and there I'm
>>> relying
>>> > on
>>> > > > Maps and Records (f.ex. union ["null","RecordType"]) in addition to
>>> > > > primitive types.
>>> > > >
>>> > > > I'm getting this error in Drill when I try to query the test file:
>>> > > >
>>> > > > Error: SYSTEM ERROR: UnsupportedOperationException: Avro union type
>>> > must
>>> > > be
>>> > > > of the format : ["null", "some-type"]
>>> > > >
>>> > > >
>>> > > > This error message provides no clues to which field are involved
>>> here
>>> > but
>>> > > > I'm hoping that this does not mean that sub structures, available
>>> in
>>> > both
>>> > > > JSON and Parquet, are not available in Avro as Records.
>>> > > >
>>> > > > Can someone please shed some light on this.
>>> > > >
>>> > > > Regards,
>>> > > > -Stefan
>>> > > >
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Kamesh.
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> Kamesh.
>>>
>>
>>
>
