[
https://issues.apache.org/jira/browse/YUNIKORN-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790991#comment-17790991
]
Peter Bacsko edited comment on YUNIKORN-2206 at 11/29/23 9:20 AM:
------------------------------------------------------------------
I'm abandoning this ticket because it's becoming really complicated.
Basically we'd need to know if the headroom Resource object is the same as the
previous one without calling {{resources.Equals()}}. This requires a very
delicate state tracking for both "headroom" and "userHeadroom", because they're
both calculated recursively by walking the queue hierarchy. At every level, we
need to examine whether something has changed (eg. allocatedResource,
maxResource, resourceUsage). If it has, then we return a boolean to the caller,
indicating the new headroom value is different. This propagates up the stack.
Finally, the original caller sees this, so it no longer uses the memoized
results and calls {{FitInMaxUndef()}} again. It's not impossible but it's very
easy to make a mistake.
For the time being, let's close it and reopen if there's a complaint about the
performance.
was (Author: pbacsko):
I'm abandoning this ticket because it's becoming really complicated.
Basically we'd need to know if the headroom Resource object is the same as the
previous one without calling {{resources.Equals()}}. This requires a very
delicate state tracking for both "headroom" and "userHeadroom", because they're
both calculated recursively by walking the queue hierarchy. At every level, we
need to examine whether something has changed (eg. allocatedResource,
maxResource, resourceUsage). If it has, then we return a boolean to caller,
indicating the new headroom value is different. Finally, the original caller
sees this, so it no longer uses the memoized results and calls
{{FitInMaxUndef()}} again. It's not impossible but it's very easy to make a
mistake.
For the time being, let's close it and reopen if there's a complaint about the
performance.
> Make user/queue headroom checks more performant
> -----------------------------------------------
>
> Key: YUNIKORN-2206
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2206
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - scheduler
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Minor
>
> Inside Application.tryAllocate() and tryReservedAllocate(), we have the
> following code:
> {noformat}
> if !userHeadroom.FitInMaxUndef(ask.GetAllocatedResource()) {
> continue
> }
> // check if this fits in the queue's headroom
> if !headRoom.FitInMaxUndef(ask.GetAllocatedResource()) {
> continue
> }
> {noformat}
> These calls are relatively expensive, but necessary. Calling them once isn't
> a problem.
> However, repeated calls are slowing things down considerably. Yunikorn keeps
> trying to schedule asks and until there's enough room for the request, these
> checks will fail. We need a way to speed it up.
> See YUNIKORN-2201 about the details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
