Yeah, try it again w/ -Dcrunch.platform=2 instead of -Dhadoop.profile=2.0 J
On Tue, Feb 25, 2014 at 11:47 AM, Bill Sparks <[email protected]> wrote: > Well I did the following and also changed the pom.xml to reference the > correct hadoop version. > > $ mvn clean install -Dhadoop.profile=2.0 -DskipTests > > <hadoop.version>2.0.6-alpha</hadoop.version> > > -- > Jonathan (Bill) Sparks > Software Architecture > Cray Inc. > > From: Josh Wills <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Tuesday, February 25, 2014 1:43 PM > To: "[email protected]" <[email protected]> > Subject: Re: Yarnchild error : crunch-0.7.0 > > Hrm-- that's usually related to the API changes between hadoop1 and > hadoop2. How did you build crunch, exactly? Did you use > -Dcrunch.platform=2? > > J > > > On Tue, Feb 25, 2014 at 11:37 AM, Bill Sparks <[email protected]> wrote: > >> Can anyone shed some light on why I would be getting the following >> error when submitting a simple crunch wordcount example. Other Hadoop MR >> applications work, just it seems that Crunch is confused about some class >> definitions. >> >> I'm running hadoop-2.0.6-alpha and have build crunch to match. >> >> Hadoop 2.0.6-alpha >> Subversion Unknown -r ca4c88898f95aaab3fd85b5e9c194ffd647c2109 >> Compiled by jenkins on 2013-10-30T07:19Z >> From source with checksum 95e88b2a9589fa69d6d5c1dbd48d4e >> >> >> 2014-02-25 13:23:00,049 INFO [main] org.apache.hadoop.mapred.MapTask: >> bufstart = 0; bufvoid = 268435456 >> 2014-02-25 13:23:00,049 INFO [main] org.apache.hadoop.mapred.MapTask: >> kvstart = 67108860; length = 16777216 >> 2014-02-25 13:23:00,070 FATAL [main] org.apache.hadoop.mapred.YarnChild: >> Error running child : java.lang.IncompatibleClassChangeError: >> org/apache/hadoop/mapreduce/JobContext.getConfiguration()Lorg/apache/hadoop/conf/Configuration; >> at >> org.apache.crunch.impl.mr.run.CrunchTaskContext.<init>(CrunchTaskContext.java:42) >> at org.apache.crunch.impl.mr.run.CrunchMapper.setup(CrunchMapper.java:37) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:757) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) >> at java.security.AccessController.doPrivileged(AccessController.java:366) >> at javax.security.auth.Subject.doAs(Subject.java:572) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153) >> >> > > > -- > Director of Data Science > Cloudera <http://www.cloudera.com> > Twitter: @josh_wills <http://twitter.com/josh_wills> >
