[
https://issues.apache.org/jira/browse/YUNIKORN-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720080#comment-17720080
]
Weiwei Yang commented on YUNIKORN-1724:
---------------------------------------
hi [~pbacsko] do you have the throughput comparison before & after applying
your patch?
the GetNewTasks() doesn't seem very expensive, as it just holds the particular
app's Rlock. Most of the time, one app's tasks get added really fast, and then
holding the read lock shouldn't be a big problem. I am guessing the expensive
part is the
[sorting|https://github.com/apache/yunikorn-k8shim/blob/cc14a81fdeb0371db861279d05fbd45a98cffe54/pkg/cache/application.go#L314-L318],
maybe a better fix is to replace the task map with a sorted map in
application.go?
> Improve the performance of shim side scheduling cycle
> -----------------------------------------------------
>
> Key: YUNIKORN-1724
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1724
> Project: Apache YuniKorn
> Issue Type: Sub-task
> Components: shim - kubernetes
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
> Labels: pull-request-available
> Attachments: getNewTasks.png
>
>
> Performance testing of Yunikorn uncovered that a lot of time is spent in
> {{Application.Schedule()}} in the shim. The problem is related to the fact
> that we collect task objects based on their state which is maintained by
> {{{}fsm.FSM{}}}. Even though we run {{Application.Schedule()}} once per
> second, it's still an issue due to the large number of {{RWMutex.RLock()}}
> calls. With a lot of pods, this consumes significant amount of CPU time.
> Also, different code paths are affected:
> The first is inside the switch-case part in {{{}Schedule(){}}}. We want to
> know the number of tasks in "New" state and we end up scanning all task
> objects for their status.
> The second is retrieving the "New" tasks from {{taskMap}} structure. This is
> done by {{GetNewTasks()}} / {{{}getTasks(){}}}, copying tasks based on their
> respective state to a new slice.
> To speed things up, we have to track the "New" tasks in a new map which is
> dynamically maintained when a new task added and when it leaves the New state
> (or the task gets removed). Knowing how many tasks we have also becomes
> trivial and won't require slice iteration/filtering.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org
