[
https://issues.apache.org/jira/browse/YARN-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820132#comment-13820132
]
Alejandro Abdelnur commented on YARN-1404:
------------------------------------------
The idea for unmanaged containers is not to modify the lifecycle of a
request/allocation/activation of container requests. This means that an
unmanaged container follow the exact same path that regular containers with the
sole exception that no process is started for it.
The {{ContainerLauncherContext}} would have a {{UNMANAGED_CONTAINER}} constant
which is an 'empty' {{ContainerLauncherContext}} instance (no environment, no
command, no local resources, etc).
If the {{UNMANAGED_CONTAINER}} constant is used as the
{{ContainerLauncherContext}} in a {{StartContainerRequest}} when doing a
{{ContainerManagementProtocol#startContainers(...)}} call, then the NodeManager
would not start the container process.
In the NodeManager, there is one {{ContainerLaunch}} instance per running
container which blocks in its {{call()}} method while the container is running.
And with the {{cleanUp()}} container method ends the container. For unmanaged
containers these 2 methods would simple use a latch instead
starting-blocking/stopping a process. By doing this, unmanaged containers will
also block as regular containers.
In addition, we need the {{ContainersMonitorImpl}} must ignore unmanaged
containers as there is not underlying process tree to monitor. This can be done
adding an {{ContainersMonitorEvent.isUnmanagedContainer()}} method that will
indicate that here is no underlying process to monitor.
> Add support for unmanaged containers
> ------------------------------------
>
> Key: YARN-1404
> URL: https://issues.apache.org/jira/browse/YARN-1404
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Affects Versions: 2.2.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: YARN-1404.patch
>
>
> Currently a container allocation requires to start a container process with
> the corresponding NodeManager's node.
> For applications that need to use the allocated resources out of band from
> Yarn this means that a dummy container process must be started.
> Impala/Llama is an example of such application which is currently starting a
> 'sleep 10y' (10 years) process as the container process. And the resource
> capabilities are used out of by and the Impala process collocated in the
> node. The Impala process ensures the processing associated to that resources
> do not exceed the capabilities of the container. Also, if the container is
> lost/preempted/killed, Impala stops using the corresponding resources.
> In addition, in the case of Llama, the current requirement of having a
> container process, gets complicates when hard resource enforcement (memory
> -ContainersMonitor- or cpu -via cgroups-) is enabled because Impala/Llama
> request resources with CPU and memory independently of each other. Some
> requests are CPU only and others are memory only. Unmanaged containers solve
> this problem as there is no underlying process with zero CPU or zero memory.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
