[
https://issues.apache.org/jira/browse/YUNIKORN-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18104318#comment-18104318
]
Manikandan R commented on YUNIKORN-3362:
----------------------------------------
Is {{requests}} map really required given that {{sortedRequests}} and
{{allocations}} maps are available? {{sortedRequests }}hold the pending asks,
where as {{allocations }}hold the allocated ones. I think we can remove
requests map if we use the appropriate structure for {{sortedRequests }}to do
READ (allocation key based retrieval) and WRITE (especially placing the ask on
the right index based on the ordering by priority followed by create time etc
and delete based on the allocation key). We do already have {{isAllocated}} of
Allocation to help us wherever needed. If we need anything more, we can add
similar one too.
> Remove asks that have been scheduled from sortedRequests
> --------------------------------------------------------
>
> Key: YUNIKORN-3362
> URL: https://issues.apache.org/jira/browse/YUNIKORN-3362
> Project: Apache YuniKorn
> Issue Type: Task
> Components: core - scheduler
> Reporter: Wilfred Spiegelenburg
> Assignee: Dale Richardson
> Priority: Major
>
> The sortedRequests inside the scheduler is a sorted list of the pending asks
> that need to be scheduled. Currently it contains all asks ever created and
> still running.
> That could cause extra overhead for the scheduling loop.
> * create a job with 10,000 pods
> * schedule 9,999 and leave them running
> * sortedRequests is 10,000 requests long
> Each time the last pod is up for scheduling the scheduler will walk over all
> requests and skip the already allocated requests. Retrieving the object and
> performing multiple checks for each request.
> As soon as the ask is allocated the ask should be removed from the list of
> sortedRequests.
> On rollback the ask must then be re-added to the list. If it does not end up
> at exactly the same point in the sorted list then that is not an issue. The
> sorting is not really fine grained as the resolution of create time is
> seconds anyway. The removal should prevent the list from growing overly long
> and the scheduler doing more work than strictly needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
