Jinghui Wang created HADOOP-11334:
-------------------------------------
Summary: Mapreduce Job Failed due to failure fetching mapper
output on the reduce side
Key: HADOOP-11334
URL: https://issues.apache.org/jira/browse/HADOOP-11334
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 2.4.1
Reporter: Jinghui Wang
Running terasort with the following options hadoop jar
hadoop-mapreduce-examples.jar terasort *-Dio.native.lib.available=false
-Dmapreduce.map.output.compress=true
-Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec*
/tmp/tera-in /tmp/tera-out
The job failed with the reducer failed to fetching the output from mappers (see
the following stacktrace). The problem is that in JIRA MAPREDUCE-1784, it added
support to handle null compressors to default to non-compressed output. In this
case, when the *io.native.lib.available* is set to true, the compressor will be
null. However, the decompressor has a Java implementation, so when the reducer
tries to read the mapper output, it uses the decompressor, but the output does
not have the Gzip header.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
