Yes that seems to have done the trick! Thanks
Lewis. On 10 September 2010 20:39, Greg Roelofs <[email protected]> wrote: > Lewis Crawford wrote: > >> I am trying unsuccessfully to apply a patch (HADOOP-6835) to hadoop-0.20.2 >> (64bit Ubuntu 10.04) > >> using ant on the command line I was able to build the project again >> and generate a new jar hadoop-0.20.3-dev-core.jar which I copied back >> into the $HADOOP_HOME and started hadoop. > > Only the jar? > >> I get the following error in the task log > >> 2010-09-10 16:27:54,706 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: >> Initializing JVM Metrics with processName=MAP, sessionId= >> 2010-09-10 16:27:55,001 INFO org.apache.hadoop.util.NativeCodeLoader: >> Loaded the native-hadoop library >> 2010-09-10 16:27:55,002 INFO >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & >> initialized native-zlib library > [...] >> 2010-09-10 16:27:55,310 FATAL org.apache.hadoop.mapred.TaskTracker: >> Error running child : java.lang.UnsatisfiedLinkError: >> org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(J)I >> at >> org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(Native >> Method) >> at >> org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(ZlibDecompressor.java:260) > > The patch modifies both the Java and the C code. Your configuration obviously > specifies native (C) code, but it looks like you didn't recompile and install > the native libraries, so you're still using the stock version. getRemaining() > is a new method in the Decompressor interface. > > I'd suggest you first simply change your config to disable native libraries > (hadoop.native.lib = false) and make sure your backported patch works OK. > 6835 supports both native and Java gzip. Then build the native code ("ant > -Dcompile.native=true mvn-install" or perhaps "ant -Dcompile.native=true jar" > --I haven't tried the latter) and install it in the "appropriate location." > (Sorry, I don't recall offhand where that is, but I'm pretty sure it's fully > documented in the LZO docs, and I recall coming across a generic native-code > twiki on the Apache Hadoop site somewhere, too.) I think that's the only > thing you're missing. > > Greg > -- ---------------------8<------------------- Keep up to date with our trip at http://lewisandjo.blogspot.com
