Dale Richardson created YUNIKORN-3386:
-----------------------------------------
Summary: Chaos scenarios for preemption under fault injection.
Key: YUNIKORN-3386
URL: https://issues.apache.org/jira/browse/YUNIKORN-3386
Project: Apache YuniKorn
Issue Type: Test
Components: core - scheduler, shim - kubernetes
Reporter: Dale Richardson
h3. Design
A tight two-queue guaranteed/max config (built as a reusable helper) so filling
the low-priority queue forces preemption when a high-priority wave arrives.
Fault
axes: bind failure on the high-priority pods, delete failure on victims (the
second
consumer of the create/delete loopback -- victim deletion goes through task
release
-> KubeClient.Delete), and node flap during the preemption window. Oracles: P1
victim accounting (a deleted victim is gone from both sides; an undeleted
victim is
characterised in full), P2 no lost pods, P3 no double-count / no node
over-capacity
at settle (O6 covers), P4 the system settles.
h3. Results
Control green with a hard non-vacuity assertion: on a full cluster the wave
triggers real PREEMPTED_BY_SCHEDULER victim deletes and low-queue usage drops
while
high-queue usage rises (a control that preempts nothing is rejected). The
delete-failure axis is red-by-design and produced the finding filed as
YUNIKORN-3382: one failed victim delete permanently wedges the preemption
(victim
stays IsPreempted, queue preemptingResource never returns to zero, the
high-priority ask starves). The node-flap axis reproduced the YUNIKORN-3375
overcommit in full (core believes the cluster is idle while pods run).
h3. Note
Red-by-design until YUNIKORN-3382 is fixed; draft per the YUNIKORN-3377
precedent.
The tight-capacity config doubles as the demonstration config for the
YUNIKORN-3375
queue-overcommit consequence.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
