[
https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219763#comment-14219763
]
Carlo Curino commented on YARN-2882:
------------------------------------
To help understand this notion, think of containers types as a priority. The
*guaranteed-start* containers, have higher priority, and are never over-booked
(i.e., when they show up in the NM they are started instantaneously). By
contrast the *queueable* containers are sent to the NM, and will be started
only when there is room in the node. Also if *guaranteed-start* containers show
up in a node that was completely utilized running *queueable* containers, the
*queueable* containers are preempted/killed, to guarantee the start of the
higher priority containers.
_(The rest of the comment below is covered in other sub-JIRAs of YARN-2877,
adding here some hints to the ideas for context)_
By having an explicit notion of container types, the AM can control when to use
one type vs the other. For example, one can use *queueable* containers for
tasks that are not yet on the critical path, and/or for short-running tasks
(higher chance to complete). One important use of *queueable* containers is to
allow us to boost utilization of the nodes (having a queue of work, minimize
the times in which the NM is idle).
> Introducing container types
> ---------------------------
>
> Key: YARN-2882
> URL: https://issues.apache.org/jira/browse/YARN-2882
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Konstantinos Karanasos
>
> This JIRA introduces the notion of container types.
> We propose two initial types of containers: guaranteed-start and queueable
> containers.
> Guaranteed-start are the existing containers, which are allocated by the
> central RM and are instantaneously started, once allocated.
> Queueable is a new type of container, which allows containers to be queued in
> the NM, thus their execution may be arbitrarily delayed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
