weichen lai created YUNIKORN-3445:
-------------------------------------
Summary: [Core] SortAllocationsBasedOnAsk prioritizes originator
and opted-out pods for preemption instead of protecting them
Key: YUNIKORN-3445
URL: https://issues.apache.org/jira/browse/YUNIKORN-3445
Project: Apache YuniKorn
Issue Type: Bug
Components: core - scheduler
Affects Versions: 1.9.0
Reporter: weichen lai
Assignee: weichen lai
h3. Overview
In {{pkg/scheduler/objects/preemption_utilities.go}},
{{SortAllocationsBasedOnAsk()}} and {{scoreAllocationBasedOnAsk()}} determine
victim ordering during preemption.
Under the current implementation, allocations with {{!allowPreemptSelf}}
(opted-out) or {{isOriginator}} (driver/manager) receive higher-order score
bits ({{scoreNoPreempt}} = {{1 << 34}}, {{scoreOriginator}} = {{1 << 33}}):
{code:go}func scoreAllocationBasedOnAsk(allocation *Allocation, ask
*resources.Resource) uint64 {
var score uint64 = 0
if allocation.IsOriginator() {
score |= scoreOriginator // 1 << 33
}
if !allocation.IsAllowPreemptSelf() {
score |= scoreNoPreempt // 1 << 34
}
score += allocation.GetAllocatedResource().TypeMatching(ask)
return score
}
{code}
Because {{SortAllocationsBasedOnAsk()}} orders allocations in descending order
({{scoreLeft > scoreRight}}), opted-out and originator allocations receive
higher scores and are placed at the beginning of the candidate slice rather
than at the end.
h2.
h3. Comparison with Design Documentation
This ordering conflicts with documented design specifications:
* **Preemption Law #1 (preemption policies are strong suggestions, not
guarantees):** Opting out of preemption and originator status are intended as a
last resort when no other suitable allocations can be preempted.
* **Quota Preemption Design Document:** The specification states that
non-originators and pods with {{allowPreemption: true}} should receive higher
scores to prioritize regular allocations for preemption over originators and
opted-out pods.
h2.
h3. Test Verification
The following test demonstrates the ordering in
{{pkg/scheduler/objects/preemption_utilities_test.go}}:
{code:go}func TestSortAllocationsBasedOnAsk_PreemptionOrdering(t *testing.T) {
node := NewNode(&si.NodeInfo{
NodeID: "node1",
SchedulableResource: &si.Resource{
Resources: map[string]*si.Quantity{"first": {Value:
100}},
},
})
res :=
resources.NewResourceFromMap(map[string]resources.Quantity{"first": 10})
total :=
resources.NewResourceFromMap(map[string]resources.Quantity{"first": 100})
ask :=
resources.NewResourceFromMap(map[string]resources.Quantity{"first": 10})
regularPod := createAllocation("regularPod", "app1", node.NodeID, true,
false, 10, false, res)
originatorPod := createAllocation("originatorPod", "app1", node.NodeID,
true, true, 10, false, res)
optedOutPod := createAllocation("optedOutPod", "app1", node.NodeID,
false, false, 10, false, res)
allocations := []*Allocation{regularPod, originatorPod, optedOutPod}
SortAllocationsBasedOnAsk(allocations, total, ask)
// Current ordering:
// allocations[0]: optedOutPod
// allocations[1]: originatorPod
// allocations[2]: regularPod
//
// Expected ordering per design: regularPod should be placed at index 0
assert.Equal(t, allocations[0].GetAllocationKey(), "regularPod",
"Expected regularPod at index 0, but got: %s",
allocations[0].GetAllocationKey())
}
{code}
*Output:*
{code}=== RUN TestSortAllocationsBasedOnAsk_PreemptionOrdering
preemption_utilities_test.go:320: Expected regularPod at index 0, but got:
optedOutPod
--- FAIL: TestSortAllocationsBasedOnAsk_PreemptionOrdering (0.00s)
{code}
h2.
h3. Impact
During Quota Preemption, reducing a queue's quota results in originator and
opted-out allocations being preempted before standard worker allocations.
h2.
h3. Proposed Solution
In {{pkg/scheduler/objects/preemption_utilities.go}}, assign the higher-order
score bits to unprotected allocations ({{allowPreemptSelf == true}} and
{{!isOriginator}}):
{code:go}const (
scoreAllowPreempt uint64 = 1 << 34
scoreNonOriginator uint64 = 1 << 33
)
func scoreAllocationBasedOnAsk(allocation *Allocation, ask *resources.Resource)
uint64 {
var score uint64 = 0
if !allocation.IsOriginator() {
score |= scoreNonOriginator
}
if allocation.IsAllowPreemptSelf() {
score |= scoreAllowPreempt
}
score += allocation.GetAllocatedResource().TypeMatching(ask)
return score
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
