Lewis Crawford wrote:
> I am trying unsuccessfully to apply a patch (HADOOP-6835) to hadoop-0.20.2
> (64bit Ubuntu 10.04)
> using ant on the command line I was able to build the project again
> and generate a new jar hadoop-0.20.3-dev-core.jar which I copied back
> into the $HADOOP_HOME and started hadoop.
Only the jar?
> I get the following error in the task log
> 2010-09-10 16:27:54,706 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
> Initializing JVM Metrics with processName=MAP, sessionId=
> 2010-09-10 16:27:55,001 INFO org.apache.hadoop.util.NativeCodeLoader:
> Loaded the native-hadoop library
> 2010-09-10 16:27:55,002 INFO
> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded &
> initialized native-zlib library
[...]
> 2010-09-10 16:27:55,310 FATAL org.apache.hadoop.mapred.TaskTracker:
> Error running child : java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(J)I
> at
> org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(Native
> Method)
> at
> org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(ZlibDecompressor.java:260)
The patch modifies both the Java and the C code. Your configuration obviously
specifies native (C) code, but it looks like you didn't recompile and install
the native libraries, so you're still using the stock version. getRemaining()
is a new method in the Decompressor interface.
I'd suggest you first simply change your config to disable native libraries
(hadoop.native.lib = false) and make sure your backported patch works OK.
6835 supports both native and Java gzip. Then build the native code ("ant
-Dcompile.native=true mvn-install" or perhaps "ant -Dcompile.native=true jar"
--I haven't tried the latter) and install it in the "appropriate location."
(Sorry, I don't recall offhand where that is, but I'm pretty sure it's fully
documented in the LZO docs, and I recall coming across a generic native-code
twiki on the Apache Hadoop site somewhere, too.) I think that's the only
thing you're missing.
Greg
