Wangda Tan commented on YARN-1011:

bq. So, hopefully we can make the policy quite configurable so that the amount 
of disfavoring can be tuned for various workloads.
I also +1 to make this policy configurable. This is way I filed YARN-4511.

Instead of threshold of "current" instant usage, I think we can use the 
threshold of "record" usage. For example, only 50% of *maximum-under-utilized* 
resource of past 30 min for each NM can be used to allocate opportunistic 
containers. This could reduce number of opportunistic containers we can 
allocate, but also this could reduce risks that opportunistic containers affect 
other containers.

bq. Can we make this per-resource? (80% memory, 120% CPU)?

> [Umbrella] Schedule containers based on utilization of currently allocated 
> containers
> -------------------------------------------------------------------------------------
>                 Key: YARN-1011
>                 URL: https://issues.apache.org/jira/browse/YARN-1011
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Arun C Murthy
>         Attachments: yarn-1011-design-v0.pdf, yarn-1011-design-v1.pdf
> Currently RM allocates containers and assumes resources allocated are 
> utilized.
> RM can, and should, get to a point where it measures utilization of allocated 
> containers and, if appropriate, allocate more (speculative?) containers.

This message was sent by Atlassian JIRA

Reply via email to