[
https://issues.apache.org/jira/browse/YUNIKORN-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wilfred Spiegelenburg resolved YUNIKORN-1715.
---------------------------------------------
Fix Version/s: 1.4.0
1.3.0
Resolution: Fixed
Closing this jira. Subtask have already been spreak over multiple releases
which makes tracking much more difficult. New issues should not be linked into
this Jira. If more needs to be done that requires an umbrella Jira we should
log a new one.
Single performance enhancements should just be tracked as single improvement
jiras.
> Yunikorn performance improvements
> ---------------------------------
>
> Key: YUNIKORN-1715
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1715
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - scheduler
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
> Labels: release-notes
> Fix For: 1.4.0, 1.3.0
>
>
> There are some methods/functions in Yunikorn which are called frequently and
> often unnecessarily. On a large, busy cluster, eliminating these calls can
> result in faster scheduling cycle, therefore better throughput.
> In the listed cases below, we can re-use a previously computed value and the
> expensive copy/sort phase can be eliminated completely.
> {*}Retrieving node iterators{*}: in
> {{{}baseNodeCollection.getNodeIteratorInternal(){}}}, we always clone the
> tree of sorted nodes, then we build a slice. The node tree is only modified
> when a node gets a new score (plus node add/removal). By reusing the sorted
> list, we avoid cloning an {{*btree.BTree}} structure and creating {{[]*Node}}
> slices.
> {*}Queue sorting{*}: only need sorting if the following occurred:
> * Allocated resource changed in one of the child queues (most common)
> * Pending resource changed from 0 to "n", or from "n" to 0 (affects
> filtering)
> * Child queue got stopped (affects filtering)
> * Child queue structure changed on config update
> {*}Application sorting{*}: in {{Queue.TryAllocate()}} and
> {{{}Queue.TryPlaceholderAllocate(){}}}, {{sortApplications()}} always runs.
> In every iteration, it calls {{Queue.GetCopyOfApps()}} then proceeds to sort
> the apps. It only has to run if something relevant happens from the sorting
> POV:
> * Application added/removed
> * Ask added to an application
> * Ask max priority changed in at least one application
> * Allocated resource changed in at least one application
> {*}Request sorting{*}: request (ask) sorting is only necessary when the
> following occurs:
> * Ask added
> * pendingAskRepeat gets 0 in an ask
> {*}Misc{*}: we can have a bunch of other stuff that helps performance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
