[
https://issues.apache.org/jira/browse/HADOOP-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated HADOOP-3581:
--------------------------------------------
Attachment: HADOOP-3581.20080904.txt
Incorporated 1 through 4.
- For (5), Moved Procfs based code in TT to a new package private class
TaskMemoryManagerThread.
- Addressed (6) by introducing two datastructures tasksToBeadded and
tasksToBeRemoved int TaskMemoryManagerThread on which addTask and removeTask
methods (called from within startNewTask and reportTaskFinished) lock.
TaskMemoryManagerThread in turns locks individually on these structures, adds
the corresponding tasks to its internal processTreeInfoMap, clears them for
future addition/removal, and then proceeds to business of memory monitoring
using processTreeInfoMap.
> Prevent memory intensive user tasks from taking down nodes
> ----------------------------------------------------------
>
> Key: HADOOP-3581
> URL: https://issues.apache.org/jira/browse/HADOOP-3581
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: Vinod Kumar Vavilapalli
> Attachments: HADOOP-3581-final.txt, HADOOP-3581.20080901.2.txt,
> HADOOP-3581.20080902.txt, HADOOP-3581.20080904.txt, HADOOP-3581.6.0.txt,
> patch_3581_0.1.txt, patch_3581_3.3.txt, patch_3581_4.3.txt,
> patch_3581_4.4.txt, patch_3581_5.0.txt, patch_3581_5.2.txt
>
>
> Sometimes user Map/Reduce applications can get extremely memory intensive,
> maybe due to some inadvertent bugs in the user code, or the amount of data
> processed. When this happens, the user tasks start to interfere with the
> proper execution of other processes on the node, including other Hadoop
> daemons like the DataNode and TaskTracker. Thus, the node would become
> unusable for any Hadoop tasks. There should be a way to prevent such tasks
> from bringing down the node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
