[
https://issues.apache.org/jira/browse/YARN-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365063#comment-16365063
]
Carlo Curino commented on YARN-7403:
------------------------------------
The v1 patch contains a much reworked version of the code. This depends on
YARN-7934 so will not compile. The key idea here is to provide algorithms that
will run every few seconds in the GPG and observe the overall state of a
federated cluster. The algorithm will leverage some of the
{{PreemptableResourceCalculator}} logic (with several additions) to compute
what is the ideal allocation for each queues. The extra effort is put into
considering "locality" among sub-clusters, which require some careful
consideration. Various heuristics can be chosen from (we implement two as
reference), and once YARN-7885/YARN-7886 will be ready to use we can experiment
on which is most closely approximating the behavior of a single
{{CapacityScheduler}} overlooking the entire federation.
> [GQ] Compute global and local "IdealAllocation"
> -----------------------------------------------
>
> Key: YARN-7403
> URL: https://issues.apache.org/jira/browse/YARN-7403
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Priority: Major
> Attachments: YARN-7403.draft.patch, YARN-7403.draft2.patch,
> YARN-7403.draft3.patch, YARN-7403.v1.patch, global-queues-preemption.PNG
>
>
> This JIRA tracks algorithmic effort to combine the local queue views of
> capacity guarantee/use/demand and compute the global amount of preemption,
> and based on that, "where" (in which sub-cluster) preemption will be enacted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
