[
https://issues.apache.org/jira/browse/MAPREDUCE-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated MAPREDUCE-1257:
-----------------------------------
Attachment: mapreduce-1257.txt
Patch adds the following counters:
|MAP_SPILLS|Map spills performed|
|MAP_MERGE_PASSES.name|Map output merge passes|
|REDUCE_MERGE_PASSES.name|Reduce input merge passes|
There were some tricky decisions about what to count as a merge pass. I decided
to count the merge that feeds directly into the reducer as a merge pass, but
didn't count intermediate in-memory merges, for example. Would appreciate
review of whether the decisions of which to count make sense, or if we should
add more counters to separate out the different cases for in-memory, on-disk,
etc.
> Ability to grab the number of spills
> ------------------------------------
>
> Key: MAPREDUCE-1257
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1257
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 0.22.0
> Reporter: Sriranjan Manjunath
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: mapreduce-1257.txt
>
>
> The counters should have information about the number of spills in addition
> to the number of spill records.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
