Optimize jobtracker's memory usage
-------------------------------------
Key: MAPREDUCE-2345
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2345
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker
Affects Versions: 0.21.0
Reporter: MengWang
Fix For: 0.23.0
To many tasks will eat up a considerable amount of JobTracker's heap space.
According to our observation, 50GB heap size can support to 5,000,000 tasks, so
we should optimize jobtracker's memory usage for more jobs and tasks. Yourkit
java profile show that counters, duplicate strings, Task waste too much memory.
Our optimization around these three points reduced jobtracker's memory to 1/3.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
