Daryn Sharp commented on HADOOP-13709:

If you catch it in the act, I bet its going to be some kind of deadlock between 
the jvm's shell reaper thread, the main process, and the new process doing the 
parseExecResult.  There's too many threads accessing highly synchronized 

I'm uneasy about this patch in general.  Changing the semantics of anything in 
hadoop-common is fraught with peril.  Subclasses are not expecting their 
parseExecResult to be run in a thread which may introduce subtle errors 
including but not limited to synchronization/volatility issues.  Or if method 
uses thread locals but now it's in a different thread, etc.

While interrupting a thread in a shell command would be nice, I think the more 
general requirement for yarn is that processes are not left running.  It's 
probably a safer assumption that no hadoop service wants lingering processes.  
Perhaps create a set of all running shell instances.  A shutdown hook could 
call destroy() on all the instances still registered.

> Clean up subprocesses spawned by Shell.java:runCommand when the shell process 
> exits
> -----------------------------------------------------------------------------------
>                 Key: HADOOP-13709
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13709
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.2.0
>            Reporter: Eric Badger
>            Assignee: Eric Badger
>         Attachments: HADOOP-13709.001.patch
> The runCommand code in Shell.java can get into a situation where it will 
> ignore InterruptedExceptions and refuse to shutdown due to being in I/O 
> waiting for the return value of the subprocess that was spawned. We need to 
> allow for the subprocess to be interrupted and killed when the shell process 
> gets killed. Currently the JVM will shutdown and all of the subprocesses will 
> be orphaned and not killed.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

Reply via email to