[
https://issues.apache.org/jira/browse/YARN-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840977#comment-13840977
]
Alejandro Abdelnur edited comment on YARN-1404 at 12/6/13 4:47 AM:
-------------------------------------------------------------------
[~vinodkv], thanks for summarizing our offline chat.
Regarding *ACLs and an on/off switch*:
IMO they are not necessary for the following reason.
You need an external system installed and running in the node to use the
resources of an unmanaged container. If you have direct access into the node to
start the external system, you are 'trusted'. If you don't have direct access
you cannot use the resources of an unmanaged container.
I think this is a very strong requirement already and it would avoid adding
code to manage a new ACL and an on/off switch.
Regarding *Liveliness*:
In the case of managed containers we don't have a liveliness 'report' and the
container process could very well be hung. In such scenario is the
responsibility of the AM to detected the liveliness of the container process
and react if it is considered hung.
In the case of unmanaged containers, the AM would have the same responsibility.
The only difference is that in the case of managed containers if the process
exits the NM detects that, while in the case of unmanaged containers this
responsibility would fall on the AM.
Because of this I think we could do without a leaseRenewal/liveliness call.
Regarding *NM assume a whole lot of things about containers* 3 bullet items:
For the my current use case none if this is needed. It could be relatively easy
to enable such functionality if a use case that needs it arises.
Regarding *Can such trusted application mix and match managed and unmanaged
containers?*:
In the way I envision how this would work, when an AM asks for a container and
gets an allocation for from the RM, the RM does not know if the AM will start a
managed or an unmanaged container. It is only between the AM and the NM that
this is known, when the ContainerLaunchContext is NULL.
Regarding *YARN-1040 should enabled starting unmanaged containers*:
If YARN-1040 would be implemented, yes, it would enable unmanaged containers.
However the scope of YARN-1040 is much bigger than unmanaged containers.
It should be also be possible implementing unmanaged containers as being
discussed and later implement YARN-1040.
Does this make sense?
was (Author: tucu00):
[~vinodkv], thanks for summarizing our offline chat.
Regarding *ACLs and an on/off switch*:
IMO they are not necessary for the following reason.
You need an external system installed and running in the node to use the
resources of an unmanaged container. If you have direct access into the node to
start the external system, you are 'trusted'. If you don't have direct access
you cannot use the resources of an unmanaged container.
I think this is a very strong requirement already and it would avoid adding a
new ACL and an on/off switch.
Regarding *Liveliness*:
In the case of managed containers we don't have a liveliness 'report' and the
container process could very well be hang. In such scenario is the
responsibility of the AM to detected the liveliness of the container process
and react if it is considered hung.
In the case of unmanaged containers, the AM would the same responsibility.
The only difference is that in the case of managed containers if the process
exits the NM detects that, while in the case of unmanaged containers this
responsibility would fall on the AM.
Because of this I think we could do without a leaseRenewal/liveliness call.
Regarding *NM assume a whole lot of things about containers* 3 bullet items:
For the my current use case none if this is needed. It could be relatively easy
to enable such functionality if a use case that needs it arises.
Regarding *Can such trusted application mix and match managed and unmanaged
containers?*:
In the way I envision how this will work, when an AM asks for a container and
gets an allocation for from the RM, the RM does not know if the AM will start a
managed or an unmanaged container. It is only between the AM and the NM that
this is known, when the ContainerLaunchContext is NULL.
Regarding *YARN-1040 should enabled starting unmanaged containers*:
If YARN-1040 would be implemented, yes, it would enable unmanaged containers.
However the scope of YARN-1040 is much bigger than unmanaged containers.
It should be also be possible implementing unmanaged containers as being
discussed and later implement YARN-1040.
Does this make sense?
> Enable external systems/frameworks to share resources with Hadoop leveraging
> Yarn resource scheduling
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-1404
> URL: https://issues.apache.org/jira/browse/YARN-1404
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Affects Versions: 2.2.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: YARN-1404.patch
>
>
> Currently Hadoop Yarn expects to manage the lifecycle of the processes its
> applications run workload in. External frameworks/systems could benefit from
> sharing resources with other Yarn applications while running their workload
> within long-running processes owned by the external framework (in other
> words, running their workload outside of the context of a Yarn container
> process).
> Because Yarn provides robust and scalable resource management, it is
> desirable for some external systems to leverage the resource governance
> capabilities of Yarn (queues, capacities, scheduling, access control) while
> supplying their own resource enforcement.
> Impala is an example of such system. Impala uses Llama
> (http://cloudera.github.io/llama/) to request resources from Yarn.
> Impala runs an impalad process in every node of the cluster, when a user
> submits a query, the processing is broken into 'query fragments' which are
> run in multiple impalad processes leveraging data locality (similar to
> Map-Reduce Mappers processing a collocated HDFS block of input data).
> The execution of a 'query fragment' requires an amount of CPU and memory in
> the impalad. As the impalad shares the host with other services (HDFS
> DataNode, Yarn NodeManager, Hbase Region Server) and Yarn Applications
> (MapReduce tasks).
> To ensure cluster utilization that follow the Yarn scheduler policies and it
> does not overload the cluster nodes, before running a 'query fragment' in a
> node, Impala requests the required amount of CPU and memory from Yarn. Once
> the requested CPU and memory has been allocated, Impala starts running the
> 'query fragment' taking care that the 'query fragment' does not use more
> resources than the ones that have been allocated. Memory is book kept per
> 'query fragment' and the threads used for the processing of the 'query
> fragment' are placed under a cgroup to contain CPU utilization.
> Today, for all resources that have been asked to Yarn RM, a (container)
> process must be started via the corresponding NodeManager. Failing to do
> this, will result on the cancelation of the container allocation
> relinquishing the acquired resource capacity back to the pool of available
> resources. To avoid this, Impala starts a dummy container process doing
> 'sleep 10y'.
> Using a dummy container process has its drawbacks:
> * the dummy container process is in a cgroup with a given number of CPU
> shares that are not used and Impala is re-issuing those CPU shares to another
> cgroup for the thread running the 'query fragment'. The cgroup CPU
> enforcement works correctly because of the CPU controller implementation (but
> the formal specified behavior is actually undefined).
> * Impala may ask for CPU and memory independent of each other. Some requests
> may be only memory with no CPU or viceversa. Because a container requires a
> process, complete absence of memory or CPU is not possible even if the dummy
> process is 'sleep', a minimal amount of memory and CPU is required for the
> dummy process.
> Because of this it is desirable to be able to have a container without a
> backing process.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
