[
https://issues.apache.org/jira/browse/YARN-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Shao updated YARN-3807:
---------------------------
Attachment: ProposalOfGuaranteedCapacitySchedulingForYARN-V1.3.pdf
> Proposal of Guaranteed Capacity Scheduling for YARN
> ---------------------------------------------------
>
> Key: YARN-3807
> URL: https://issues.apache.org/jira/browse/YARN-3807
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler, fairscheduler
> Reporter: Wei Shao
> Attachments: ProposalOfGuaranteedCapacitySchedulingForYARN-V1.3.pdf
>
>
> This proposal talks about limitations of the YARN scheduling policies for SLA
> applications, and tries to solve them by YARN-3806 and the new scheduling
> policy called guaranteed capacity scheduling.
> Guaranteed capacity scheduling makes guarantee to the applications that they
> can get resources under specified capacity cap in totally predictable manner.
> The application can meet SLA more easily since it is self-contained in the
> shared cluster - external uncertainties are eliminated.
> For example, suppose queue A has initial capacity 100G memory, and there are
> two pending applications 1 and 2, 1’s specified capacity is 70G, 2’s
> specified capacity is 50G. Queue A may accept application 1 to run first and
> makes guarantee that 1 can get resources exponentially up to its capacity and
> won’t be preempted (if allocation of 1 is 5G in scheduling cycle N, demand is
> 80G, exponential factor is 2. In N+1, it can get 5G, in N+2, it can get 10G,
> in N+3, it can get 20G, and in N+4, it can get 30G, reach its capacity).
> Later, when the cluster is free, queue A may decide to scale up by increasing
> its capacity to 120G, so it can accept application 2 and make guarantee to it
> as well. Queue A can scale down to its initial capacity when any application
> completes.
> Guaranteed capacity scheduling also has other features that the example
> doesn’t illustrate. See proposal for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
