Carlo Curino commented on YARN-7403:

The v1 patch contains a much reworked version of the code. This depends on 
YARN-7934 so will not compile. The key idea here is to provide algorithms that 
will run every few seconds in the GPG and observe the overall state of a 
federated cluster. The algorithm will leverage some of the 
{{PreemptableResourceCalculator}} logic (with several additions) to compute 
what is the ideal allocation for each queues. The extra effort is put into 
considering "locality" among sub-clusters, which require some careful 
consideration. Various heuristics can be chosen from (we implement two as 
reference), and once YARN-7885/YARN-7886 will be ready to use we can experiment 
on which is most closely approximating the behavior of a single 
{{CapacityScheduler}} overlooking the entire federation.


> [GQ] Compute global and local "IdealAllocation"
> -----------------------------------------------
>                 Key: YARN-7403
>                 URL: https://issues.apache.org/jira/browse/YARN-7403
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: federation
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>            Priority: Major
>         Attachments: YARN-7403.draft.patch, YARN-7403.draft2.patch, 
> YARN-7403.draft3.patch, YARN-7403.v1.patch, global-queues-preemption.PNG
> This JIRA tracks algorithmic effort to combine the local queue views of 
> capacity guarantee/use/demand and compute the global amount of preemption, 
> and based on that, "where" (in which sub-cluster) preemption will be enacted.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to