Can you post a JIRA around the example avro schema and the poor error message?
-- Jacques Nadeau CTO and Co-Founder, Dremio On Sat, Oct 17, 2015 at 2:20 AM, Stefán Baxter <[email protected]> wrote: > Hi, > > This turned out to be a incomplete union: > > > - {"name": "market", "type": ["string"]} > > Avro tools ignored it and created the classes as if this was a normal > string field. > > I would really appreciate it if the error messages Drill produces would be > structured in a more helpful way. > > Knowing what went wrong is fine, knowing why it went wrong is even better > and I think that in this case it should be easy to include a piece of the > header that is being parse/processed. > > Regards, > -Stefan > > > > On Fri, Oct 16, 2015 at 4:45 PM, Stefán Baxter <[email protected]> > wrote: > > > Hi Kamesh, > > > > I pulled, made and deployed 1.2-SNAPSHOT (master/head/"Latest commit > > 12fb9a > > < > https://github.com/apache/drill/commit/12fb9ac11e25a2bbd9fff5d9cdb6a355b96ffadb > > > > 3 days ago") producing the same results. > > > > I can not see a RC3 Branch, am I missing something? > > > > Regards, > > -Stefan > > > > > > > > > > On Fri, Oct 16, 2015 at 4:31 PM, Stefán Baxter < > [email protected]> > > wrote: > > > >> Hi Kamesh, > >> > >> I must be lagging behind then, let me double check. > >> > >> Regards, > >> -Stefan > >> > >> On Fri, Oct 16, 2015 at 4:23 PM, Kamesh <[email protected]> > wrote: > >> > >>> Hi Stefan, > >>> Do you have latest code changes with you (1.2-SNAPSHOT)? Could you > >>> please > >>> try RC3 build. > >>> Yes, it supports nested records. > >>> > >>> > >>> On Fri, Oct 16, 2015 at 8:28 PM, Stefán Baxter < > >>> [email protected]> > >>> wrote: > >>> > >>> > Hi, > >>> > > >>> > Sorry I forgot to mention that I'm already using a 1.2-SNAPSHOT > >>> > > >>> > Should it be supporting Records as a data type? (use to map sub > >>> > structures). > >>> > > >>> > Regards, > >>> > -Stefán > >>> > > >>> > > >>> > > >>> > > >>> > On Fri, Oct 16, 2015 at 2:31 PM, Kamesh <[email protected]> > >>> wrote: > >>> > > >>> > > Hi Stefan, > >>> > > We have enhanced Drill Avro Reader by supporting complex data > types > >>> like > >>> > > Union, Map etc.. This is targeted for 1.2.0 release. This release > >>> will > >>> > > happen in a day or in a couple days. > >>> > > > >>> > > On Fri, Oct 16, 2015 at 6:30 PM, Stefán Baxter < > >>> > [email protected]> > >>> > > wrote: > >>> > > > >>> > > > Hi, > >>> > > > > >>> > > > I'm changing our logging to use the Avro format and there I'm > >>> relying > >>> > on > >>> > > > Maps and Records (f.ex. union ["null","RecordType"]) in addition > to > >>> > > > primitive types. > >>> > > > > >>> > > > I'm getting this error in Drill when I try to query the test > file: > >>> > > > > >>> > > > Error: SYSTEM ERROR: UnsupportedOperationException: Avro union > type > >>> > must > >>> > > be > >>> > > > of the format : ["null", "some-type"] > >>> > > > > >>> > > > > >>> > > > This error message provides no clues to which field are involved > >>> here > >>> > but > >>> > > > I'm hoping that this does not mean that sub structures, available > >>> in > >>> > both > >>> > > > JSON and Parquet, are not available in Avro as Records. > >>> > > > > >>> > > > Can someone please shed some light on this. > >>> > > > > >>> > > > Regards, > >>> > > > -Stefan > >>> > > > > >>> > > > >>> > > > >>> > > > >>> > > -- > >>> > > Kamesh. > >>> > > > >>> > > >>> > >>> > >>> > >>> -- > >>> Kamesh. > >>> > >> > >> > > >
