yangwwei commented on a change in pull request #323:
URL:
https://github.com/apache/incubator-yunikorn-core/pull/323#discussion_r709671683
##########
File path: pkg/scheduler/objects/nodesorting.go
##########
@@ -42,33 +48,79 @@ func (fairnessNodeSortingPolicy) PolicyType()
policies.SortingPolicy {
return policies.FairnessPolicy
}
-func (binPackingNodeSortingPolicy) ScoreNode(node *Node) float64 {
- // choose most loaded node first
- return resources.LargestUsageShare(node.GetAvailableResource())
+func absResourceUsage(node *Node, weights *map[string]float64) float64 {
+ totalWeight := float64(0)
+ usage := float64(0)
+
+ shares := node.GetResourceUsageShares()
+ for k, v := range shares {
+ weight, found := (*weights)[k]
+ if !found || weight == float64(0) {
+ continue
+ }
+ if math.IsNaN(v) {
+ continue
+ }
+ usage += v * weight
+ totalWeight += weight
+ }
+
+ var result float64
+
+ if totalWeight == float64(0) {
+ result = float64(0)
+ } else {
+ result = usage / totalWeight
+ }
+ return result
Review comment:
can we normalize this value int? I do not think we need that much
accuracy using float64. Similar to
https://github.com/kubernetes/kubernetes/blob/dd5c3a109e2f156774576b7e7a38e104fa5c7202/pkg/scheduler/framework/interface.go#L100,
it has a MaxNodeScore defined.
##########
File path: pkg/scheduler/partition.go
##########
@@ -149,7 +149,7 @@ func (pc *PartitionContext) updateNodeSortingPolicy(conf
configs.PartitionConfig
log.Logger().Info("NodeSorting policy set from config",
zap.String("policyName", configuredPolicy.String()))
}
-
pc.nodes.SetNodeSortingPolicy(objects.NewNodeSortingPolicy(conf.NodeSortPolicy.Type))
+
pc.nodes.SetNodeSortingPolicy(objects.NewNodeSortingPolicy(conf.NodeSortPolicy.Type,
conf.NodeSortPolicy.ResourceWeights))
Review comment:
Can we add some validation in the `configvalidator.go`?
I think we should check the weight >=0 and <=100. This helps because if the
user sets some invalid value, we can find this early when we load the configs
from the configmap.
We might also want to print a WARN if we found user provided weights do not
contain vcore and memory.
##########
File path: pkg/scheduler/objects/node.go
##########
@@ -552,6 +552,22 @@ func (sn *Node) UnReserveApps() ([]string, []int) {
return appReserve, askRelease
}
+// Gets map of name -> resource usages per type in shares (0 to 1). Can return
NaN.
+func (sn *Node) GetResourceUsageShares() map[string]float64 {
+ sn.RLock()
+ defer sn.RUnlock()
+ res := make(map[string]float64)
+ used := resources.Add(sn.allocatedResource, sn.occupiedResource)
Review comment:
can we calculate this based on the existing `availableResource`?
so we do not need some extra computation to calculate `used` here
##########
File path: pkg/scheduler/objects/nodesorting.go
##########
@@ -42,33 +48,79 @@ func (fairnessNodeSortingPolicy) PolicyType()
policies.SortingPolicy {
return policies.FairnessPolicy
}
-func (binPackingNodeSortingPolicy) ScoreNode(node *Node) float64 {
- // choose most loaded node first
- return resources.LargestUsageShare(node.GetAvailableResource())
+func absResourceUsage(node *Node, weights *map[string]float64) float64 {
Review comment:
we need a better name for this method, this is calculating the node
score, maybe we can call it `calculateNodeScore()`.
##########
File path: pkg/scheduler/objects/node.go
##########
@@ -552,6 +552,22 @@ func (sn *Node) UnReserveApps() ([]string, []int) {
return appReserve, askRelease
}
+// Gets map of name -> resource usages per type in shares (0 to 1). Can return
NaN.
+func (sn *Node) GetResourceUsageShares() map[string]float64 {
+ sn.RLock()
+ defer sn.RUnlock()
+ res := make(map[string]float64)
+ used := resources.Add(sn.allocatedResource, sn.occupiedResource)
+ if sn.totalResource == nil {
+ // no resources present, so no usage
+ return res
+ }
+ for k, v := range sn.totalResource.Resources {
Review comment:
I think we should iterate the used resources instead of total here, node
total resources can have more resource types than used. E.g total has A, B, C,
D, and used only has A, B. Can we cover this scenario in UT as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
