Enforce RSS memory limit in TaskMemoryManagerThread
---------------------------------------------------
Key: MAPREDUCE-1181
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1181
Project: Hadoop Map/Reduce
Issue Type: New Feature
Components: tasktracker
Affects Versions: 0.20.1
Reporter: Scott Chen
Fix For: 0.20.1
TaskMemoryManagerThread will periodically check the rss memory usage of every
task. If the memory usage exceeds the specified threshold, the task will be
killed. Also if the total rss memory of all tasks exceeds (total amount of
memory - specified reserved memory). The task with least progress will be
killed to recover the reserved rss memory.
This is similar to the virtual memory limit provided by
TaskMemoryManagerThread. But now the limit is for rss memory. This new feature
allow us to avoid page swapping which is prone to error.
The following are the related configurations
mapreduce.reduce.memory.rss.mb // RSS memory allowed for a reduce task
mapreduce.map.memory.rss.mb // RSS memory allowed for a map task
mapreduce.tasktracker.reserved.memory.rss.mb // RSS memory reserved (not
for tasks) on a tasktracker
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
