A mapper should use some heuristics to decide whether to run the combiner
during spills
---------------------------------------------------------------------------------------
Key: HADOOP-2960
URL: https://issues.apache.org/jira/browse/HADOOP-2960
Project: Hadoop Core
Issue Type: Bug
Reporter: Runping Qi
Right now, the combiner, if set, will be called for each spill, no mapper
whether the combiner can actually reduce the values.
The mapper should use some heuristics to decide whether to run the combiner
during spills.
One of such heuristics is to check the the ratio of the nymber of keys to the
number of unique keys in the spill.
The combiner will be called only if that ration exceeds certain threshold (say
2).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
