[
https://issues.apache.org/jira/browse/YUNIKORN-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Craig Condit updated YUNIKORN-847:
----------------------------------
Description:
After YUNIKORN-807, node sorting uses a scoring algorithm which uses the
largest resource share to determine priority. However, resource units are not
taken into account, and consequently, either fair or binpacking tends to look
almost exclusively at the largest absolute values for units. In some
environments this can lead to priority inversions.
We should instead calculate in-use percentages for each resource, and then
weight them to arrive at an overall usage percentage, which can then be used
for scoring. Weights can be controlled at the partition level.
Configuration is as follows:
{code:java}
partitions:
- name: default
nodesortpolicy:
type: fair
resourceweights:
vcore: 4.0
memory: 1.0
queues:
- name: root
submitacl: '*'
{code}
In this example, *vcore* is weighted double what memory is. If
*resourceweights* is not specified, the configuration is equivalent to:
{code}
{ vcore: 1, memory: 1}
{code}
For example, in the default configuration, if a node has 60% of its vcores in
use, and 20% of its memory in use, node usage would be calculated at 40% using
equal weights. In the example given above, usage would be calculated as 52%.
was:
After YUNIKORN-807, node sorting uses a scoring algorithm which uses the
largest resource share to determine priority. However, resource units are not
taken into account, and consequently, either fair or binpacking tends to look
almost exclusively at the largest absolute values for units. In some
environments this can lead to priority inversions.
We should instead calculate in-use percentages for each resource, and then
weight them to arrive at an overall usage percentage, which can then be used
for scoring. Weights can be controlled at the partition level.
Configuration is as follows:
{code:java}
partitions:
- name: default
nodesortpolicy:
type: fair
resourceweights:
vcore: 4.0
memory: 1.0
queues:
- name: root
submitacl: '*'
{code}
In this example, *vcore* is weighted double what memory is. If
*resourceweights* is not specified, the configuration is equivalent to *{
vcore: 1, memory: 1}.*
For example, in the default configuration, if a node has 60% of its vcores in
use, and 20% of its memory in use, node usage would be calculated at 40% using
equal weights. In the example given above, usage would be calculated as 52%.
> Use resource weighting for node sorting
> ---------------------------------------
>
> Key: YUNIKORN-847
> URL: https://issues.apache.org/jira/browse/YUNIKORN-847
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - scheduler
> Reporter: Craig Condit
> Assignee: Craig Condit
> Priority: Major
> Labels: pull-request-available
>
> After YUNIKORN-807, node sorting uses a scoring algorithm which uses the
> largest resource share to determine priority. However, resource units are not
> taken into account, and consequently, either fair or binpacking tends to look
> almost exclusively at the largest absolute values for units. In some
> environments this can lead to priority inversions.
> We should instead calculate in-use percentages for each resource, and then
> weight them to arrive at an overall usage percentage, which can then be used
> for scoring. Weights can be controlled at the partition level.
> Configuration is as follows:
> {code:java}
> partitions:
> - name: default
> nodesortpolicy:
> type: fair
> resourceweights:
> vcore: 4.0
> memory: 1.0
> queues:
> - name: root
> submitacl: '*'
> {code}
> In this example, *vcore* is weighted double what memory is. If
> *resourceweights* is not specified, the configuration is equivalent to:
> {code}
> { vcore: 1, memory: 1}
> {code}
>
> For example, in the default configuration, if a node has 60% of its vcores in
> use, and 20% of its memory in use, node usage would be calculated at 40%
> using equal weights. In the example given above, usage would be calculated as
> 52%.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
