[
https://issues.apache.org/jira/browse/HADOOP-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Gummadi updated HADOOP-2721:
---------------------------------
Attachment: HADOOP-2721-v2.patch
Attaching an intermediate patch HADOOP-2721-v2.patch that addresses:
(1) As we are moving pidfiles beyond the scope of memory management, we should
move the pidfile related api(getPidFilePath, removePidFile) out of
TaskMemoryManager.
Moved them to TaskTracker.
(2) destroySubprocessTree() should also ensure that the process-tree is indeed
terminated/killed by sending SIGKILL if needed. We should do something in the
line of ProcfsBasedProcessTree.SigKillThread.
destroySubprocessTree() ensures that the process-tree is indeed
terminated/killed by sending SIGKILL. SigKillThread is moved from
ProcfsBasedProcessTree to a separate file.
(3) I think the check to ensure that the process is the process group leader
should be made always. This adds one more level of confidence that we are
indeed killing the right process. But this should be done in
destroySubprocessTree() and only when used by TT. In other cases, this may not
be the case. So we need to separate this may be by having separate api for
destroySession and destroySubprocessTree. TT would use destroySession.
The check to ensure that the process is the process group leader is made
always. Now we have destroyProcessGroup() instead of destroySubProcessTree().
Testcase: Increased the memoryLimit to 30MB so that the subtree would be
bigger(now it has 8 processes). Added a verification in
TestProcfsBasedProcessTree that checks if all the sub-processes are indeed
wiped off.
> Use job control for tasks (and therefore for pipes and streaming)
> -----------------------------------------------------------------
>
> Key: HADOOP-2721
> URL: https://issues.apache.org/jira/browse/HADOOP-2721
> Project: Hadoop Core
> Issue Type: Improvement
> Components: contrib/streaming, mapred, pipes
> Reporter: Owen O'Malley
> Assignee: Ravi Gummadi
> Attachments: HADOOP-2721-v2.patch, HADOOP-2721.patch,
> HADOOP-2721.patch, HADOOP-2721.patch
>
>
> We should use the setsid command when the task is launched to create a new
> session. We should be able to use the setsid program when we launch the bash
> process to create a new session. That will allow us to kill the entire
> session with a single signal and remove the need for the ping methods in both
> TaskTracker.Child and pipes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
