Hi guys, Thanks the speedy replies.
I will try Harsh's suggestion and see if this solves the issue, if not I will just do what Michael suggested and replace the jars on each of the nodes. Thanks again! On Wed, Oct 3, 2012 at 12:39 PM, Harsh J <[email protected]> wrote: > Hi Ben, > > As long as the switch of libraries doesn't impact the execution of the > Child task code itself, for Apache Hadoop 1.x, using the config > "mapreduce.user.classpath.first" set to true may solve your trouble. > > On Wed, Oct 3, 2012 at 4:51 PM, Ben Rycroft <[email protected]> wrote: > > Hi all, > > > > I have a jar that uses the Hadoop API to launch various remote mapreduce > > jobs (ie, im not using the command-line to initiate the job). The service > > jar that executes the various jobs is built with maven's > > "jar-with-dependencies". > > > > My jobs all run fine except one that uses commons-codec 1.7, I get: > > > > FATAL org.apache.hadoop.mapred.Child: Error running child : > > java.lang.NoSuchMethodError: > > > org.apache.commons.codec.binary.Base64.encodeAsString([B)Ljava/lang/String; > > > > I think this is because my jar is including commons-codec 1.7 whereas my > > Hadoop install's lib has commons-codec 1.4 ... > > > > Is their any way to instruct Hadoop to use the distributed commons-codec > 1.7 > > (I assume this is distributed as a job dependency) rather than the > > commons-codec 1.4 in the hadoop 1.0.3 core lib? > > > > Removing commons-codec-1.4.jar from my Hadoop library folder did seem to > > solve the problem for a bit, but is not working on another VM. Replacing > the > > 1.4 jar with the 1.7 does seem to fix the problem but this doesn't seem > too > > sane. Hopefully there is a better alternative. > > > > Thanks! > > > > -- > Harsh J >
