Kishan Ghetia created YUNIKORN-3448:
---------------------------------------
Summary: Revert autoscaling state when a scale-up-triggered ask
loses queue headroom
Key: YUNIKORN-3448
URL: https://issues.apache.org/jira/browse/YUNIKORN-3448
Project: Apache YuniKorn
Issue Type: Improvement
Components: core - scheduler, shim - kubernetes
Reporter: Kishan Ghetia
A pod can become ineligible for scheduling after it has already triggered
cluster autoscaling.
Timeline:
1. A pod fits its queue and user headroom, but no existing node can accommodate
it.
2. YuniKorn sends FAILED, the shim marks the pod Unschedulable, and Karpenter
begins provisioning capacity.
3. YuniKorn sets scaleUpTriggered=true.
4. Before the new node becomes Ready, another allocation consumes the remaining
queue, parent-queue, or user headroom.
5. The original pod no longer fits headroom, but YuniKorn does not reset
scaleUpTriggered or change the pod’s Unschedulable status.
6. Karpenter can consequently provision a node that cannot be used by the
triggering pod. The unused node may later be consolidated.
The expected over-provisioning is likely small, but this can create unnecessary
capacity and node churn.
Proposed behavior:
When a scale-up-triggered ask no longer fits queue or user headroom:
- Reset its scaleUpTriggered state.
- Send an appropriate update to the shim so the pod is no longer presented to
the autoscaler as Unschedulable.
- Allow the pod to trigger autoscaling again if headroom later becomes
available and no suitable node exists.
Acceptance criteria:
- Cover the transition from autoscaling-eligible to headroom-ineligible.
- Update both the core ask state and Kubernetes pod status.
- Verify that the pod can trigger autoscaling again after regaining headroom.
- Add a test where competing allocations consume headroom while capacity is
being provisioned.
- Preserve existing outstanding-request headroom accounting.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
