Vinod Kumar Vavilapalli updated YARN-2518:
    Labels:   (was: container dispatch in-process job node)

Today's containers in YARN are necessarily process-trees - meaning they are 
separate processes with their own command-lines, environment etc. If you want 
an 'in-process' container, given NMs are JVMs, you need native support for some 
sort of a first-class JVM container in YARN. This doesn't exist today, so this 
ticket doesn't make much sense as of now.

Your use-case is valid. And as the above comment says, the solution is to 
launch a container and reuse it.

One other relevant work you may be interested in is YARN-896.

> Support in-process container executor
> -------------------------------------
>                 Key: YARN-2518
>                 URL: https://issues.apache.org/jira/browse/YARN-2518
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager
>    Affects Versions: 2.5.0
>         Environment: Linux, Windows
>            Reporter: BoYang
>            Priority: Minor
> Node Manage always creates a new process for a new application. We have hit a 
> scenario where we want the node manager to execute the application inside its 
> own process, so we get fast response time. It would be nice if Node Manager 
> or YARN can provide native support for that.
> In general, the scenario is that we have a long running process which can 
> accept requests and process the requests inside its own process. Since YARN 
> is good at scheduling jobs, we want to use YARN to dispatch jobs (e.g. 
> requests in JSON) to the long running process. In that case, we do not want 
> YARN container to spin up a new process for each request. Instead, we want 
> YARN container to send the request to the long running process for further 
> processing.

This message was sent by Atlassian JIRA

Reply via email to