[
https://issues.apache.org/jira/browse/YARN-10174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766648#comment-17766648
]
Shilun Fan commented on YARN-10174:
-----------------------------------
Thank you very much for bringing up this JIRA. Here's my understanding, and I
look forward to further discussing it with you:
Currently, YARN Federation supports the allocation and scheduling of jobs to
various subclusters based on Router Policy and AMRM Policy. For example:
Consider the queue root.a, which is associated with three subclusters: SC-1,
SC-2, and SC-3. In our current setup, the Router Policy is defined as SC-1:0.2,
SC-2:0.5, SC-3:0.3, and the AMRM Policy is SC-1:0.3, SC-2:0.4, SC-3:0.3. This
is a fixed configuration.
However, we may have various types of jobs, such as AdHoc jobs or SLA jobs, and
the predefined rules may not necessarily align well with the specific needs of
these jobs. Therefore, when submitting jobs from the client side, it would be
beneficial to express job preferences.
In this JIRA, we will be adding support for configuring multiple weight
combinations for a specific queue. For example:
Router Policy:
SLA: SC-1:1, SC-2:0, SC-3:0
AdHoc: SC-1:0.2, SC-2:0.5, SC-3:0.3
Default: SC-1:0.2, SC-2:0.5, SC-3:0.3
When clients submit tasks, they can specify their task type in the tag.
> Add colored policies to enable manual load balancing across sub clusters
> ------------------------------------------------------------------------
>
> Key: YARN-10174
> URL: https://issues.apache.org/jira/browse/YARN-10174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Young Chen
> Assignee: zhengchenyu
> Priority: Major
>
> Add colored policies to enable manual load balancing across sub clusters
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
