Use of protocol schemas in IDLs is optional. On Fri, Feb 21, 2014 at 5:52 AM, Software Dev <[email protected]> wrote: > I should also note that the IDL introduced another concept that we > previously were dealing with... protocol. Is the protocol just a grouping of > related records? > > FYI, we are using Avro strictly for serialization/de-serialization and no > RPC features. > > > On Thu, Feb 20, 2014 at 4:19 PM, Software Dev <[email protected]> > wrote: >> >> We have a similar use case and I would like to just "flatten" out the >> schema by including the fields from the parent into the child without >> nesting. The reason I don't want to nest is because it doesnt play well with >> some of our tools (Impala, Pig, etc) >> >> Now back to your initial response. I started playing around with the IDL >> but I still can't seem to figure out how to inherit all the fields from a >> parent into a child record. >> >> I would like the final "Chile" record to look like: >> >> { >> "type": "record", >> "name": "Child", >> "fields" : [ >> { >> "name": "bar", >> "type": "string" >> }, >> { >> "name": "foo", >> "type": "string" >> } >> ] >> } >> >> >> On Thu, Feb 20, 2014 at 12:55 PM, Lewis John Mcgibbney >> <[email protected]> wrote: >>> >>> Going back to your initial question... why don't you wish to include >>> "foo" within nested "parent"? >>> I am not quite getting it here. >>> In my case, the nested records were of substantial size including dozens >>> of fields, which then had nested records. It did not scale to write out AVSC >>> definitions for the data model. >>> >>> >>> On Thu, Feb 20, 2014 at 8:28 PM, Software Dev <[email protected]> >>> wrote: >>>> >>>> Thanks for the input. I'm guessing then the above problem can only >>>> solved with IDL and not AVSC? >>>> >>>> >>>> On Thu, Feb 20, 2014 at 11:45 AM, Lewis John Mcgibbney >>>> <[email protected]> wrote: >>>>> >>>>> Hey, >>>>> Did you check out the IDL documentation? >>>>> http://avro.apache.org/docs/current/idl.html >>>>> I had similar data modeling issues a while back and this helped out A >>>>> LOT. >>>>> hth >>>>> >>>>> >>>>> On Thu, Feb 20, 2014 at 6:58 PM, Software Dev >>>>> <[email protected]> wrote: >>>>>> >>>>>> Is there anyway to include the fields of another schema into our >>>>>> schema WITHOUT it creating a nested record? >>>>>> >>>>>> >>>>>> { >>>>>> "type": "record", >>>>>> "name": "Parent", >>>>>> "fields" : [ >>>>>> { >>>>>> "name": "foo", >>>>>> "type": "string" >>>>>> } >>>>>> ] >>>>>> } >>>>>> >>>>>> { >>>>>> "type": "record", >>>>>> "name": "Child", >>>>>> "fields" : [ >>>>>> { >>>>>> "name": "bar", >>>>>> "type": "string" >>>>>> }, >>>>>> // I dont want it nested like this >>>>>> // { >>>>>> // "name": "parent", >>>>>> // "type": "Parent" >>>>>> // } >>>>>> ] >>>>>> } >>>>>> >>>>>> So in this example is there a way to have child include both the "bar" >>>>>> field as well as "foo" without it nested under parent? >>>>>> >>>>>> Thanks >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Lewis >>>> >>>> >>> >>> >>> >>> -- >>> Lewis >> >> >
-- Harsh J
