I am afraid I have pulled the Avro upgrade request far too soon. I should have checked my mailbox first.
We have been using avro-1.7.0-rc0 for a week now to resolve AVRO-1046<https://issues.apache.org/jira/browse/AVRO-1046> issue. No problems were observed so far but we have not tested it with any other Hadoop version but chd3. Cheers, Chris On Tue, Jun 12, 2012 at 10:47 PM, Tom White <[email protected]> wrote: > Funny - I was just writing an email about potential dragons... > > Before upgrading someone should check that Avro 1.7.0 works with > released versions of Hadoop. In the past there have been problems with > Avro and dependencies like Jackson conflicting with versions that > Hadoop uses. In particular, the MR classpath can be controlled via the > configuration property mapreduce.user.classpath.first and the env > property HADOOP_USER_CLASSPATH_FIRST. By setting these to true MR will > use the newer Avro libraries, however there is a risk that Hadoop will > not work with the newer versions. > > Cheers, > Tom > > On Tue, Jun 12, 2012 at 3:36 PM, Josh Wills <[email protected]> wrote: > > On Tue, Jun 12, 2012 at 1:28 PM, Gabriel Reid <[email protected]> > wrote: > >> On Tue, Jun 12, 2012 at 10:14 PM, Josh Wills <[email protected]> > wrote: > >>> > >>> I saw Chris' tweet that Avro 1.7.0 was released-- thoughts on > >>> upgrading? Does it buy us any good stuff we want? > >> > >> The fix of ReflectDatumReader not working correctly with Specific > >> Records (https://issues.apache.org/jira/browse/AVRO-1046) would allow > >> us to remove a fair bit of Avro code that works around that bug -- if > >> we do do the upgrade, I'd certainly volunteer to weed out those > >> workarounds. > >> > >> On the other hand, I just did a quick scan of the release notes for > >> 1.7.0 ( > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310911&version=12318848 > ) > >> and I didn't notice anything that would really be a big reason to > >> upgrade right away. > >> > >> Are there any reasons *not* to upgrade (other than risks of something > >> else being broken)? Maybe the cleanup of the Avro code that we can do > >> is reason enough to do the upgrade. > > > > +tom explicitly > > > > I remember having a bunch of frustrations with 1.6.0 and having to > > stay on 1.5.4 for longer than I wanted because of some critical bugs > > that didn't get fixed until 1.6.2, but I also think the move from > > 1.5.4 to 1.6.0 involved a much larger rewrite than what I see from the > > release notes for 1.7.0. Tom White is traveling across the US right > > now, but I'm wondering if he has a feel for whether 1.7.0 is likely to > > contain any dragons. :) > > > >> > >> - Gabriel > > > > > > > > -- > > Director of Data Science > > Cloudera > > Twitter: @josh_wills >
