[
https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15171114#comment-15171114
]
Karthik Kambatla commented on YARN-1011:
----------------------------------------
bq. So we are going to add promotion notification to the AM RM protocol, right?
By corollary, we would be adding a flag to the initial allocation that shows if
it was guaranteed or opportunistic, right?
Yes and yes.
bq. Its very likely that an app may have a guaranteed and an opportunistic
container. And when it gives up a guaranteed container then we will need to
allocate another one to it. That may be the opportunistic container or a new
one.
Yes, particularly when it has unmet demand outside of the opportunistic
allocation. If it has no other demand, theoretically we should promote the
opportunistic container. Promoting across nodes is not always desirable. I feel
we should let the app tell us what to do. If it absolutely wants a guaranteed
container, we should allocate a guaranteed container and kill the opportunistic
one. If it does not want, we can let the opportunistic container continue to
run.
bq. The relationship between overall cluster utilization and node
over-allocation is not clear. Like you say, for an under allocated cluster, it
would likely be easy to find a guaranteed container. So I am not sure if we
should go ahead and make this tenuous link formal by adding it as a config in
the code. Regardless of cluster allocation state, a node could be
over-allocated.
The overall cluster utilization is an implementation detail, its sole purpose
is to reduce the chances of running into cases that need cross-node promotion.
Node over-allocation continues to be the primary configuration admins fiddle
with.
> [Umbrella] Schedule containers based on utilization of currently allocated
> containers
> -------------------------------------------------------------------------------------
>
> Key: YARN-1011
> URL: https://issues.apache.org/jira/browse/YARN-1011
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Arun C Murthy
> Attachments: patch-for-yarn-1011.patch, yarn-1011-design-v0.pdf,
> yarn-1011-design-v1.pdf, yarn-1011-design-v2.pdf
>
>
> Currently RM allocates containers and assumes resources allocated are
> utilized.
> RM can, and should, get to a point where it measures utilization of allocated
> containers and, if appropriate, allocate more (speculative?) containers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
