Hi Juwei
What is the value for mapred.output.compression.codec? It'd be
better to determine whether the output files are compressed by getting the
codec of the same and not just from the size of files.
Regards
Bejoy.K.S
On Fri, Feb 17, 2012 at 12:07 PM, Juwei Shi <shiju...@gmail.com> wrote:
> Hi,
>
> I am benchmarking the cluster using the Terasort package of Hadoop 0.20.2.
> I enabled compression for both map output (*mapred.compress.map.output*)
> and reduce output (*mapred.output.compress*). I checked the parameter in
> Job.xml, both are true. I can see that the compression for Map output
> works, but it seems that the compression for reduce output does not work.
> The output of the job on HDFS is also 1TB.
>
> Thanks!
>
> - Juwei
>
