Wei Shao updated YARN-3807:
    Attachment:     (was: 

> Proposal of Guaranteed Capacity Scheduling for YARN
> ---------------------------------------------------
>                 Key: YARN-3807
>                 URL: https://issues.apache.org/jira/browse/YARN-3807
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler, fairscheduler
>            Reporter: Wei Shao
>         Attachments: ProposalOfGuaranteedCapacitySchedulingForYARN-V1.2.pdf
> This proposal talks about limitations of the YARN scheduling policies for SLA 
> applications, and tries to solve them by YARN-3806 and the new scheduling 
> policy called guaranteed capacity scheduling.
> Guaranteed capacity scheduling makes guarantee to the applications that they 
> can get resources under specified capacity cap in totally predictable manner. 
> The application can meet SLA more easily since it is self-contained in the 
> shared cluster - external uncertainties are eliminated.
> For example, suppose queue A has initial capacity 100G memory, and there are 
> two pending applications 1 and 2, 1’s specified capacity is 70G, 2’s 
> specified capacity is 50G. Queue A may accept application 1 to run first and 
> makes guarantee that 1 can get resources exponentially up to its capacity and 
> won’t be preempted (if allocation of 1 is 5G in scheduling cycle N, demand is 
> 80G, exponential factor is 2. In N+1, it can get 5G, in N+2, it can get 10G, 
> in N+3, it can get 20G, and in N+4, it can get 30G, reach its capacity). 
> Later, when the cluster is free, queue A may decide to scale up by increasing 
> its capacity to 120G, so it can accept application 2 and make guarantee to it 
> as well. Queue A can scale down to its initial capacity when any application 
> completes.
> Guaranteed capacity scheduling also has other features that the example 
> doesn’t illustrate. See proposal for more details.

This message was sent by Atlassian JIRA

Reply via email to