Hi, I am trying unsuccessfully to apply a patch (HADOOP-6835) to hadoop-0.20.2 (64bit Ubuntu 10.04)
I have downloaded the tar.gz and can build the project - I tried to apply the patch from https://issues.apache.org/jira/browse/HADOOP-6835 (specifically HADOOP-6835.v9.yahoo-0.20.2xx-branch.patch) and there is an issue with the diff -git for src/core/org/apache/hadoop/io/compress/GzipCodec.java however I thought I had resolved this by working through the patch diffs. using ant on the command line I was able to build the project again and generate a new jar hadoop-0.20.3-dev-core.jar which I copied back into the $HADOOP_HOME and started hadoop. on running a test map reduce task using streaming bin/hadoop jar contrib/streaming/hadoop-0.20.2-streaming.jar -input /gzip -output /out -mapper cat -reducer wc I get the following error in the task log 2010-09-10 16:27:54,706 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2010-09-10 16:27:55,001 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library 2010-09-10 16:27:55,002 INFO org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library 2010-09-10 16:27:55,004 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 1 2010-09-10 16:27:55,012 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100 2010-09-10 16:27:55,110 INFO org.apache.hadoop.mapred.MapTask: data buffer = 79691776/99614720 2010-09-10 16:27:55,110 INFO org.apache.hadoop.mapred.MapTask: record buffer = 262144/327680 2010-09-10 16:27:55,171 INFO org.apache.hadoop.streaming.PipeMapRed: PipeMapRed exec [/bin/cat] 2010-09-10 16:27:55,243 INFO org.apache.hadoop.streaming.PipeMapRed: R/W/S=1/0/0 in:NA [rec/s] out:NA [rec/s] 2010-09-10 16:27:55,244 INFO org.apache.hadoop.streaming.PipeMapRed: Records R/W=1/1 2010-09-10 16:27:55,244 INFO org.apache.hadoop.streaming.PipeMapRed: MRErrorThread done 2010-09-10 16:27:55,245 INFO org.apache.hadoop.streaming.PipeMapRed: MROutputThread done 2010-09-10 16:27:55,245 INFO org.apache.hadoop.streaming.PipeMapRed: mapRedFinished 2010-09-10 16:27:55,310 FATAL org.apache.hadoop.mapred.TaskTracker: Error running child : java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(J)I at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(Native Method) at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.getRemaining(ZlibDecompressor.java:260) at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:93) at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:76) at java.io.InputStream.read(InputStream.java:85) at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134) at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:136) at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:40) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) Any thoughts or pointers how to apply the patch gratefully received. Thanks, Lewis.
