[
https://issues.apache.org/jira/browse/HADOOP-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628304#action_12628304
]
Devaraj Das commented on HADOOP-3581:
-------------------------------------
Some comments:
1) fix the bug in setTaskMemoryManagerEnabled that unconditionally sets the
flag taskMemoryManagerEnabled to true
2) Increase the default monitoring interval to 5 seconds. 300 msecs seems
really low
3) Change processTreeInfo to processTreeInfoMap. The name is confusing.
4) Move the PID file cleanup to cleanup of Task.
5) Move the Procfs based code to a different class outside the TaskTracker.
5) The thread doing the memory monitoring is locking quite a big section of
code where interaction with the OS is also involved. The same lock is also used
for adding tasks to be monitored before their launch. This might negatively
impact fast start of tasks.
> Prevent memory intensive user tasks from taking down nodes
> ----------------------------------------------------------
>
> Key: HADOOP-3581
> URL: https://issues.apache.org/jira/browse/HADOOP-3581
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: Vinod Kumar Vavilapalli
> Attachments: HADOOP-3581-final.txt, HADOOP-3581.20080901.2.txt,
> HADOOP-3581.20080902.txt, HADOOP-3581.6.0.txt, patch_3581_0.1.txt,
> patch_3581_3.3.txt, patch_3581_4.3.txt, patch_3581_4.4.txt,
> patch_3581_5.0.txt, patch_3581_5.2.txt
>
>
> Sometimes user Map/Reduce applications can get extremely memory intensive,
> maybe due to some inadvertent bugs in the user code, or the amount of data
> processed. When this happens, the user tasks start to interfere with the
> proper execution of other processes on the node, including other Hadoop
> daemons like the DataNode and TaskTracker. Thus, the node would become
> unusable for any Hadoop tasks. There should be a way to prevent such tasks
> from bringing down the node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
