[
https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097690#comment-15097690
]
Wangda Tan commented on YARN-1011:
----------------------------------
Thanks for updating docs, [~kasha]. Most part of the design doc makes sense to
me, I have the same concern regarding SLA of opportunistic containers:
I'm not very familiar with CPU scheduling in CGroup. Do you know what will
happen for the following example:
Let's say a node (strict resource usage disabled) has 60% of CPU utilization,
which consumed by 10 processes, each of them has CPU share = 1024. Now there's
a new process comes with CPU share = 1, it wants 100% of CPU, will the system
gives 40% of the idle CPU to the new process or the new process can only get
small proportion of the rest idle resources?
If the process can full leverage idle resources regardless of CPU share, I'm
fine with setting lowest CPU-share for opportunistic containers.
bq. Won't the scheduler just try to assign it to that application again? Seems
like they'll fight one another. "Here's a container".."No, I don't like this
container".."No really, you should like this container, take it."....
I think this is a valid concern, [~kasha], do you think scheduler should smart
enough to avoid allocating opportunistic containers to an app who doesn't want
such opportunistic container?
> [Umbrella] Schedule containers based on utilization of currently allocated
> containers
> -------------------------------------------------------------------------------------
>
> Key: YARN-1011
> URL: https://issues.apache.org/jira/browse/YARN-1011
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Arun C Murthy
> Attachments: yarn-1011-design-v0.pdf, yarn-1011-design-v1.pdf,
> yarn-1011-design-v2.pdf
>
>
> Currently RM allocates containers and assumes resources allocated are
> utilized.
> RM can, and should, get to a point where it measures utilization of allocated
> containers and, if appropriate, allocate more (speculative?) containers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
