option 3 is not available to us. i have been using option 1 without issues so far (except in hive), but i have only worked with MR1 old api and avro- generic so far (not sure which of these is relevant...).
On Sat, Oct 20, 2012 at 3:59 AM, Jacob Metcalf <[email protected]>wrote: > > Yes I have CDH4 working happily with Avro 1.7.0 following the process > described below. > > The various methods of distributing the Avro jar are discussed in > http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job. > I > could not distribute the newer Avro jar via methods 1 and 2 because of > classpath issues (discussed in AVRO-1103) so settled for the not so > satisfactory option 3. > > The latest version of Avro has a patch for AVRO-1103 and a Maven profile > to compile against the newer versions of Hadoop. I have not tried upgrading > yet, but you should definitely try this first before going down the self > patching route. > > Regards > > Jacob > > ------------------------------ > From: [email protected] > Subject: Re: version of avro > Date: Fri, 19 Oct 2012 13:16:24 -0700 > To: [email protected] > > > > On Oct 19, 2012, at 1:03pm, Koert Kuipers wrote: > > i noticed avro version 1.5.4 is included with some version/distros of > hadoop and hive... is there a reason why 1.5.4 is included specifically and > not newer ones? are there some incompatibilities to be aware of? i would > like to use a newer version > > > In the mail archives there was a discussion back in July about using Avro > 1.7 with CHD4. where Jacob Metcalf said: > > Avro 17 is compiled against Hadoop 1 i.e. CDH3. > > I have it running aginst CDH4 but I had to patch and recompile Avro, then > replace the Avro 153 in the Hadoop lib directory with 17. I have attached > my patch against JIRA AVRO-1103 for consideration. > > > It sounds like if you're using CDH3/Hadoop 1.0 then you should be able to > use Avro 1.7 as-is, but I haven't tried this myself. > > -- Ken > > -------------------------------------------- > http://about.me/kkrugler > +1 530-210-6378 > > > > >
