Jeff, your Hadoop native libraries appear to be loading correctly, but the libz you have installed on your system cannot decode the header of the .gz file. This is the line of code causing it:
stream = init(this.header.windowBits()); and the init() function is native. RB On 2/26/08, Jeff Eastman <[EMAIL PROTECTED]> wrote: > > I'm processing a number of .gz compressed Apache and other logs using > Hadoop 0.15.2 and encountering fatal decompression errors such as: > > > > 08/02/26 12:09:12 INFO mapred.JobClient: Task Id : > task_200802171116_0001_m_000005_0, Status : FAILED > > java.lang.InternalError > > at > org.apache.hadoop.io.compress.zlib.ZlibDecompressor.init(Native Method) > > at > org.apache.hadoop.io.compress.zlib.ZlibDecompressor.<init>(ZlibDecompres > sor.java:111) > > at > org.apache.hadoop.io.compress.GzipCodec.createDecompressor(GzipCodec.jav > a:188) > > at > org.apache.hadoop.io.compress.GzipCodec.createInputStream(GzipCodec.java > :170) > > at > org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:7 > 5) > > at > org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat > .java:50) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:156) > > at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1787) > > > > I looked in Jira but did not find any issues. Is this pilot error? Some > of the files work just fine. Is there a workaround besides unzipping all > the files in the DFS? > > > > > Jeff > > -- ------------------------------- Riccardo Boscolo, PhD V.P. of Core Technology Netseer Inc. 11943 Montana Ave, Suite 200 Los Angeles, CA 90049 T: 310-597-4482 F: 310-597-4489 Email: [EMAIL PROTECTED] -------------------------------
