[
https://issues.apache.org/jira/browse/HADOOP-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
he yongqiang updated HADOOP-4845:
---------------------------------
Attachment: Hadoop-4845.patch
make the size field of MapOutput count compressed bytes size for both
shuffleToDisk and shuffleInMemory mapoutputs.
the size field was previously used to record the decompressed byte size for
shuffleInMemory mapoutputs and had the same vaule as MapOutput.data.length, but
this size field is only used one time to deteremine if the mapoutput's size is
zero or not. I think it is ok to use the size filed to record the compressed
size of the MapOutput and use the data.length as the size of decommpressed
bytes.
For avoiding confusing, i changed the name size to compressedSize.
> Shuffle counter issues
> ----------------------
>
> Key: HADOOP-4845
> URL: https://issues.apache.org/jira/browse/HADOOP-4845
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.20.0
> Reporter: Chris Douglas
> Assignee: he yongqiang
> Fix For: 0.20.0
>
> Attachments: Hadoop-4845.patch
>
>
> HADOOP-4749 added a new counter tracking the bytes shuffled into the reduce.
> It adds an accumulator to ReduceCopier instead of simply incrementing the new
> counter and did not define a human-readable value in
> src/mapred/org/apache/hadoop/mapred/Task_Counter.properties.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
