[
https://issues.apache.org/jira/browse/YARN-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874338#comment-13874338
]
Michael Lv commented on YARN-1609:
----------------------------------
Hi [~hitesh], what is missing IMO is that we do not have a good way currently
in YARN, specifically in NodeManager, to allow a user land daemon to manage its
own process, and in the mean time, those processes, daemon or not, are also
managed by ContainerManager. The key here is to allow daemon service container
to own the lifecycle of the task/process it spawns and monitors.
Those daemon or node level services, could be implemented as auxiliary
services, but in that way they would be out side of resource allocation, and
execution/tracking by YARN. If those daemon process can be supported in NM as a
first class citizen, many of today's workaround/hacky code won't be needed.
> Add Service Container type to NodeManager in YARN
> -------------------------------------------------
>
> Key: YARN-1609
> URL: https://issues.apache.org/jira/browse/YARN-1609
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.2.0
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: Add Service Container type to NodeManager in YARN-V1.pdf
>
>
> From our work to support running OpenMPI on YARN (MAPREDUCE-2911), we found
> that it’s important to have framework specific daemon process manage the
> tasks on each node directly. The daemon process, most likely similar in other
> frameworks as well, provides critical services to tasks running on that
> node(for example “wireup”, spawn user process in large numbers at once etc).
> In YARN, it’s hard, if not possible, to have the those processes to be
> managed by YARN.
> We propose to extend the container model on NodeManager side to support
> “Service Container” to run/manage such framework daemon/services process. We
> believe this is very useful to other application framework developers as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
