[
https://issues.apache.org/jira/browse/HADOOP-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627625#action_12627625
]
Hemanth Yamijala commented on HADOOP-3581:
------------------------------------------
Looks good.
Few minor comments:
{{ProcFsBasedProcessTree}}
- allProcessInfo is being used only within getProcessTree, hence it can be a
local variable. Otherwise, it will hold memory unnecessarily
- processTree should be cleared before every re-construction of the process
tree.
- isAlive(pid): If there's an IOException, the code needs to return false.
{{TaskMemoryMonitorThread}}
- IMO, addTaskToMonitor is a better name than createProcessTreeInfo, as it
better conveys the meaning of the method. Likewise removeTaskToMonitor should
replace removeProcessTreeInfo.
{{hadoop-default.xml}}
- I think the documentation need not include details about the classes etc, but
should make sense to the administrator. Something like: The interval, in
milliseconds, the tasktracker waits between two cycles of monitoring a task's
memory usage.
- Also, generally the framework doesn't seem to define configuration variables
with upper case, so maybe
mapred.tasktracker.taskmemorymanager.monitoring-interval ?
- Millisecs should be expanded to milliseconds.
> Prevent memory intensive user tasks from taking down nodes
> ----------------------------------------------------------
>
> Key: HADOOP-3581
> URL: https://issues.apache.org/jira/browse/HADOOP-3581
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: Vinod Kumar Vavilapalli
> Attachments: HADOOP-3581-final.txt, HADOOP-3581.20080901.2.txt,
> HADOOP-3581.6.0.txt, patch_3581_0.1.txt, patch_3581_3.3.txt,
> patch_3581_4.3.txt, patch_3581_4.4.txt, patch_3581_5.0.txt, patch_3581_5.2.txt
>
>
> Sometimes user Map/Reduce applications can get extremely memory intensive,
> maybe due to some inadvertent bugs in the user code, or the amount of data
> processed. When this happens, the user tasks start to interfere with the
> proper execution of other processes on the node, including other Hadoop
> daemons like the DataNode and TaskTracker. Thus, the node would become
> unusable for any Hadoop tasks. There should be a way to prevent such tasks
> from bringing down the node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
