[
https://issues.apache.org/jira/browse/YUNIKORN-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17407164#comment-17407164
]
Chia-Ping Tsai commented on YUNIKORN-807:
-----------------------------------------
[~ccondit] Thanks for this nice patch and sorry for late comment.
It seems to me a small behavior change is introduced by this patch. This patch
scores node according to `resources.LargestUsageShare`. However, the largest
usage in our cluster is always storage (and all nodes have same storage
capacity). It results in the order is always based on `node id`. Before this
patch, the order will compare remaining "usage" (vcore, memory, etc) when
largest usage is equal.
There are two possible solutions.
# the nodeScore of nodeRef should keep all usages ([]float64) and the
implementation of `Less` function should reuse `resources.CompareShares`
# add an new config to enable users to control scope. for example, the
following config means we should compare vcore instead of largest resource
{code:java}
nodesortpolicy:
type: binpacking
usage: vcore
{code}
That’s just my two cents :)
> Improve performance of node sorting
> -----------------------------------
>
> Key: YUNIKORN-807
> URL: https://issues.apache.org/jira/browse/YUNIKORN-807
> Project: Apache YuniKorn
> Issue Type: New Feature
> Components: core - scheduler
> Reporter: Craig Condit
> Assignee: Craig Condit
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Attachments: Node Sorting Performance Improvement.pdf
>
>
> YuniKorn currently sorts all nodes on demand whenever scheduling of a
> container occurs. This causes significant performance degradation as the
> number of nodes increases.
> If we replace the on-demand sorting with a B-Tree sorted proactively, we can
> improve performance considerably.
> This is a similar approach to YUNIKORN-21, but without the associated
> behavioral changes.
> I've attached a design document with the details of the approach and the
> performance improvement gained.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
