[
https://issues.apache.org/jira/browse/YUNIKORN-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790716#comment-17790716
]
Peter Bacsko edited comment on YUNIKORN-2206 at 11/29/23 6:02 AM:
------------------------------------------------------------------
My idea: cache Resource-Resource pairs which failed the comparison. It's safe
to assume that headroom values don't change very quickly. If a pair is found
inside the cache, then it means the given ask doesn't fit.
Basically, it's a {{map[*resource.Resource]map[*Resource.Resource]bool}}. Key
is the headroom value, value is a map that contains other resource objects
which are smaller.
Cache would be per application for easier cleanup and maintenance. Contents can
be deleted inside {{cleanUpAsks()}}.
A global cache is also possible, stale entries are removed periodically. We
check when a particular Resource-Resource pair was last accessed and after a
while, we remove the entries.
EDIT: realized this doesn't work, because we'll always have a different
Resource headroom object, even if the contents are the same, so can't use it as
a map key. Have to think further...
was (Author: pbacsko):
My idea: cache Resource-Resource pairs which failed the comparison. It's safe
to assume that headroom values don't change very quickly. If a pair is found
inside the cache, then it means the given ask doesn't fit.
Basically, it's a {{map[*resource.Resource]map[*Resource.Resource]bool}}. Key
is the headroom value, value is a map that contains other resource objects
which are smaller.
Cache would be per application for easier cleanup and maintenance. Contents can
be deleted inside {{cleanUpAsks()}}.
A global cache is also possible, stale entries are removed periodically. We
check when a particular Resource-Resource pair was last accessed and after a
while, we remove the entries.
EDIT: realized this doesn't work, because we'll always have a different
Resource object for the key, even if the contents are the same, so can't use it
as a map key. Have to think further...
> Make user/queue headroom checks more performant
> -----------------------------------------------
>
> Key: YUNIKORN-2206
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2206
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - scheduler
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
>
> Inside Application.tryAllocate() and tryReservedAllocate(), we have the
> following code:
> {noformat}
> if !userHeadroom.FitInMaxUndef(ask.GetAllocatedResource()) {
> continue
> }
> // check if this fits in the queue's headroom
> if !headRoom.FitInMaxUndef(ask.GetAllocatedResource()) {
> continue
> }
> {noformat}
> These calls are relatively expensive, but necessary. Calling them once isn't
> a problem.
> However, repeated calls are slowing things down considerably. Yunikorn keeps
> trying to schedule asks and until there's enough room for the request, these
> checks will fail. We need a way to speed it up.
> See YUNIKORN-2201 about the details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
