Add counters to show number of key/values that have been sorted and merged in
the maps and reduces
--------------------------------------------------------------------------------------------------
Key: HADOOP-2774
URL: https://issues.apache.org/jira/browse/HADOOP-2774
Project: Hadoop Core
Issue Type: Bug
Reporter: Owen O'Malley
For each *pass* of the sort and merge, I would like a count of the number of
records. So for example, if the map output 100 records and they were sorted
once, the counter would be 100. If it spilled twice and was merged together, it
would be 200. Clearly in a multi-level merge, it may not be a multiple of the
number of map output records. This would let the users easily see if they have
values like io.sort.mb or io.sort.factor set too low.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
