[
https://issues.apache.org/jira/browse/YARN-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13875418#comment-13875418
]
Hitesh Shah commented on YARN-1609:
-----------------------------------
[~lvm] I think I understand what the goals are though I am still unclear on the
specifics as to what the real problems were when trying to launch processes
directly via the NM as compared to using the service container.
- Is the NM flow too slow for launching a new container or killing one?
- Is it difficult to monitoring of the status of a container?
- Can the NM not handle launching/managing a large no. of containers on a
single machine?
It would also be helpful to understand what kind of workarounds/hacks you
needed to do so as to understand the underlying problems that you faced.
> Add Service Container type to NodeManager in YARN
> -------------------------------------------------
>
> Key: YARN-1609
> URL: https://issues.apache.org/jira/browse/YARN-1609
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.2.0
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: Add Service Container type to NodeManager in YARN-V1.pdf
>
>
> From our work to support running OpenMPI on YARN (MAPREDUCE-2911), we found
> that it’s important to have framework specific daemon process manage the
> tasks on each node directly. The daemon process, most likely similar in other
> frameworks as well, provides critical services to tasks running on that
> node(for example “wireup”, spawn user process in large numbers at once etc).
> In YARN, it’s hard, if not possible, to have the those processes to be
> managed by YARN.
> We propose to extend the container model on NodeManager side to support
> “Service Container” to run/manage such framework daemon/services process. We
> believe this is very useful to other application framework developers as well.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
