On Fri, Feb 8, 2013 at 10:34 AM, David Medinets <[email protected]>wrote:
> I am running a map-reduce job. As soon as my mapper tried to serialize > a Mutation I run into a NoSuchMethodError in reference to > FieldValueMetaData. I could simply delete the hue-plugins jar file but > that seems inelegant to me. When running a mapper can I shift when the > jar files are loaded? Would HADOOP_USER_CLASSPATH_FIRST help in this > situation? What about adding the libthift jar file to the > sun.boot.class.path property? > I found a likely answer by searching for "hadoop distributedcache precedence" which located p. 163 of Hadoop: The Definitive Guide. It mentions this issue specifically under "Task Classpath Precedence." On the client side, you can force Hadoop to put the user classpath first by setting HADOOP_USER_CLASSPATH_FIRST to true. However, for a task classpath, you should set "mapreduce.task.classpath.first" to true. A bit more searching leads me to believe that this property name is different in different versions of Hadoop. I just checked 1.0.3 and it appears to be "mapreduce.user.classpath.first" there. This question indicates there could be at least two other possible values: http://stackoverflow.com/questions/11685949/overriding-default-hadoop-jars-in-class-path Apparently you should check the static property names in org.apache.hadoop.mapred.TaskRunner to find the appropriate property name for your version of Hadoop. Let us know if that works! Billie
