[
https://issues.apache.org/jira/browse/YARN-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated YARN-2518:
------------------------------------------
Labels: (was: container dispatch in-process job node)
Today's containers in YARN are necessarily process-trees - meaning they are
separate processes with their own command-lines, environment etc. If you want
an 'in-process' container, given NMs are JVMs, you need native support for some
sort of a first-class JVM container in YARN. This doesn't exist today, so this
ticket doesn't make much sense as of now.
Your use-case is valid. And as the above comment says, the solution is to
launch a container and reuse it.
One other relevant work you may be interested in is YARN-896.
> Support in-process container executor
> -------------------------------------
>
> Key: YARN-2518
> URL: https://issues.apache.org/jira/browse/YARN-2518
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Affects Versions: 2.5.0
> Environment: Linux, Windows
> Reporter: BoYang
> Priority: Minor
>
> Node Manage always creates a new process for a new application. We have hit a
> scenario where we want the node manager to execute the application inside its
> own process, so we get fast response time. It would be nice if Node Manager
> or YARN can provide native support for that.
> In general, the scenario is that we have a long running process which can
> accept requests and process the requests inside its own process. Since YARN
> is good at scheduling jobs, we want to use YARN to dispatch jobs (e.g.
> requests in JSON) to the long running process. In that case, we do not want
> YARN container to spin up a new process for each request. Instead, we want
> YARN container to send the request to the long running process for further
> processing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
