[
https://issues.apache.org/jira/browse/MAPREDUCE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
MengWang updated MAPREDUCE-2345:
--------------------------------
Description: Too many tasks will eat up a considerable amount of
JobTracker's heap space. According to our observation, 50GB heap size can
support to 5,000,000 tasks, so we should optimize jobtracker's memory usage for
more jobs and tasks. Yourkit java profile show that counters, duplicate
strings, task waste too much memory. Our optimization around these three points
reduced jobtracker's memory to 1/3. (was: To many tasks will eat up a
considerable amount of JobTracker's heap space. According to our observation,
50GB heap size can support to 5,000,000 tasks, so we should optimize
jobtracker's memory usage for more jobs and tasks. Yourkit java profile show
that counters, duplicate strings, Task waste too much memory. Our optimization
around these three points reduced jobtracker's memory to 1/3. )
> Optimize jobtracker's memory usage
> -------------------------------------
>
> Key: MAPREDUCE-2345
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2345
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.21.0
> Reporter: MengWang
> Labels: hadoop
> Fix For: 0.23.0
>
> Attachments: jt-memory-useage.bmp
>
>
> Too many tasks will eat up a considerable amount of JobTracker's heap space.
> According to our observation, 50GB heap size can support to 5,000,000 tasks,
> so we should optimize jobtracker's memory usage for more jobs and tasks.
> Yourkit java profile show that counters, duplicate strings, task waste too
> much memory. Our optimization around these three points reduced jobtracker's
> memory to 1/3.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
