Implement a memory-to-memory sort in the map task
-------------------------------------------------
Key: HADOOP-5870
URL: https://issues.apache.org/jira/browse/HADOOP-5870
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Reporter: Arun C Murthy
The motivation is similar to HADOOP-5831...
Currently we collect map-outputs in the sort buffer (io.sort.mb) which we
eventually sort and spill to disk. For latency-sensitive applications with
sufficient memory, e.g. terasort, we could do better by doing a
memory-to-memory sort followed by a final memory-to-disk merge.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
