[
https://issues.apache.org/jira/browse/HADOOP-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated HADOOP-3581:
--------------------------------------------
Status: Open (was: Patch Available)
Cancelling patch to incorporate Devaraj's review comments offline.
- Remove all references to ProcfsBasedProcessTree in TT. TT should strictly
use the abstract interface ProcessTree. Only. So that implementing and using
some other processTree doesn't imply any changes to TT.
- Let the pidFile be written during task creation *only if* memory management
is enabled. Despite this, hitting the disk to write pid to a file for each task
might affect short jobs.
- TT should get an instance of ProcessTree from a configuration variable -
mapred.tasktracker.processtreeimpl
- Make TT use a string for a pid instead of an integer.
> Prevent memory intensive user tasks from taking down nodes
> ----------------------------------------------------------
>
> Key: HADOOP-3581
> URL: https://issues.apache.org/jira/browse/HADOOP-3581
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: Vinod Kumar Vavilapalli
> Attachments: patch_3581_0.1.txt, patch_3581_3.3.txt,
> patch_3581_4.3.txt, patch_3581_4.4.txt, patch_3581_5.0.txt
>
>
> Sometimes user Map/Reduce applications can get extremely memory intensive,
> maybe due to some inadvertent bugs in the user code, or the amount of data
> processed. When this happens, the user tasks start to interfere with the
> proper execution of other processes on the node, including other Hadoop
> daemons like the DataNode and TaskTracker. Thus, the node would become
> unusable for any Hadoop tasks. There should be a way to prevent such tasks
> from bringing down the node.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
