[
https://issues.apache.org/jira/browse/YUNIKORN-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18107825#comment-18107825
]
Wilfred Spiegelenburg commented on YUNIKORN-3406:
-------------------------------------------------
added you to the contributors list and assigned the Jira to you, welcome!
> Support queue-headroom-based limits for application unschedulable backoff
> -------------------------------------------------------------------------
>
> Key: YUNIKORN-3406
> URL: https://issues.apache.org/jira/browse/YUNIKORN-3406
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - scheduler
> Reporter: Kishan Ghetia
> Assignee: Kishan Ghetia
> Priority: Major
>
> Application.tryAllocate() currently stops after a fixed number of
> unschedulable asks. For example with config:
> application.unschedasks.backoff: "50"
> application.unschedasks.backoff.delay: "5s"
> With large Spark executor bursts, the queue may have quota for more than 50
> asks, but later asks are not marked {{SchedulingAttempted}} before the
> application backs off. Consequently, the outstanding-request scan cannot mark
> those pods {{{}Unschedulable{}}}, delaying autoscaling even though queue
> quota is available.
> Proposal: add an option for {{tryAllocate()}} to use the remaining queue
> quota/headroom as the unschedulable-ask limit. It should attempt enough
> sorted requests to fill the available queue quota, using the same effective
> limit as outstanding-request marking, and then apply the existing application
> backoff.
> This preserves the backoff’s protection against unbounded node and predicate
> checks while exposing all autoscaling demand that the queue can currently
> accept.
> Acceptance criteria:
> * When queue headroom supports more than the fixed count, enough asks are
> attempted to fill that headroom.
> * Asks beyond available queue headroom are not exposed for autoscaling.
> * Existing backoff behavior remains after the quota-based limit is reached.
> * Tests cover more than 50 asks and mixed request sizes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
