On Fri, 13 Jan 2012 13:59:12 -0800 (PST), vvkbtnkr <[email protected]> wrote: > I am running a hadoop jar and keep getting this error - > java.lang.NoSuchMethodError: > org.codehaus.jackson.JsonParser.getValueAsLong()
Nobody seems to have answered this while I was on vacation, so... Okay, here's what I know, having run into a similar problem before. Hadoop, at some point, decided that it wanted to use Jackson for JSON serialization as an alternative method of serialization for its Configuration objects. Of course, this means that Hadoop now depends on Jackson, and in particular some specific version of Jackson. In my own version (CDH3b3 = Hadoop 0.20.2+737), this is 1.5.2 The $HADOOP_HOME/lib directory in each of my tasktracker nodes contains: jackson-core-asl-1.5.2.jar jackson-mapper-asl-1.5.2.jar So? So here's how the Hadoop classloader works: first it loads up anything in $HADOOP_HOME/lib, then it loads anything on your distributed classpath, including the specified job JAR (we're using an überJAR containing all of our dependencies). But, like all classloaders... THE FIRST VERSION OF A LOADED CLASS TAKES PRECEDENCE This is Very Important for Java to work right, and should be the first thing any Java hacker unfortunate enough to have to understand classloaders should learn. In your case, this means that you're trying to use the 1.8.x version of org.codehaus.jackson.JsonParser, but the 1.5.2 version has already been loaded, and that takes precedence. Since JsonParser doesn't have a method called getValueAsLong() in Jackson 1.5.2, you get an error. So, what do you do? You've got a couple of options. If you control your own deployment environment, that's great; you can try upgrading Hadoop's version of Jackson. It's pretty good about backwards-compatibility, so you probably won't have any problems. On the other hand, if you don't control your deployment environment (or you don't want to risk fiddling with it, which I Totally Get), you can try to rewrite your code to use Jackson version 1.5.2; that's what we did, and it's sort of awkward in some of the ways it does things, but I think for your purposes it probably won't be too difficult. I think JsonParser.getLongValue() is the right method to consider. HTH
