[
https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085738#comment-15085738
]
Karthik Kambatla commented on YARN-1011:
----------------------------------------
bq, Just to make sure I understand. When you say max threshold < 1 are you
saying an NM could not advertise 48 vcores if there are only 24 vcores
physically available?
You can continue to advertise more vcores.
Consider a cluster with nodes of 1 physical core. Let us say each node
advertises 10 *vcores*. Today, let us say your CPU utilization under these
settings is 50% running 10 containers. All these containers in this context
would be GUARANTEED containers. I am proposing we set a max threshold for the
RM over-allocating containers to 95%.This essentially means, the RM allocates
OPPORTUNISTIC containers on this node (that has been previously fully
allocated) until we hit the utilization threshold of 95% - say, running 19
containers. At this point if one container's usage goes higher taking us beyond
95%, we kill enough OPPORTUNISTIC containers to bring this under 95%. May be,
the max allowed threshold could be higher - 99%. I am wary of setting it to
100% unless we have some other way of differentiating "running comfortably at
100%" vs "contention at 100%" because both look the same. Also, I am assuming
people would be very happy with 95% utilization if we achieve that :)
bq. nodes can be at 100% CPU, 100% Network, or 100% Disk for long periods of
time (several minutes). Memory could get to something like 80% before
corrective action would be required.
I am beginning to see the need for different thresholds for different
resources. While I wouldn't necessarily shoot for 100, I can see someone
configuring it to 95% CPU, 85% network (as this could spike significantly with
shuffle etc.), 90% disk, 80% memory. And, we would stop over-allocating the
moment we hit *any one* of these thresholds.
Should we keep it simple to begin with and have one config, and add other
configs in the future? Or, do you think the config-per-resource should be there
from the get go?
> [Umbrella] Schedule containers based on utilization of currently allocated
> containers
> -------------------------------------------------------------------------------------
>
> Key: YARN-1011
> URL: https://issues.apache.org/jira/browse/YARN-1011
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Arun C Murthy
> Attachments: yarn-1011-design-v0.pdf, yarn-1011-design-v1.pdf
>
>
> Currently RM allocates containers and assumes resources allocated are
> utilized.
> RM can, and should, get to a point where it measures utilization of allocated
> containers and, if appropriate, allocate more (speculative?) containers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
