[
https://issues.apache.org/jira/browse/HADOOP-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655912#action_12655912
]
Runping Qi commented on HADOOP-4845:
------------------------------------
It's ok to record the counters for successful reduces.
If "shuffleInMemory", you should have the numbers of bytes for both the
compressed data and uncompressed data.
We should always use the numbers for compressed data.
Estimate on the compression ratio is very useful.
It can be approximated based on the numbers for the ShuffleInMemory data.
> Shuffle counter issues
> ----------------------
>
> Key: HADOOP-4845
> URL: https://issues.apache.org/jira/browse/HADOOP-4845
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.20.0
> Reporter: Chris Douglas
> Fix For: 0.20.0
>
>
> HADOOP-4749 added a new counter tracking the bytes shuffled into the reduce.
> It adds an accumulator to ReduceCopier instead of simply incrementing the new
> counter and did not define a human-readable value in
> src/mapred/org/apache/hadoop/mapred/Task_Counter.properties.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
