[
https://issues.apache.org/jira/browse/YARN-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17424902#comment-17424902
]
Andras Gyori commented on YARN-10795:
-------------------------------------
Closing this epic, as all of its subtasks are done. With these improvements, I
have been able to reduce a reinit with 5000+ queues from 4 mins to few seconds.
> Improve Capacity Scheduler reinitialisation performance
> -------------------------------------------------------
>
> Key: YARN-10795
> URL: https://issues.apache.org/jira/browse/YARN-10795
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Reporter: Andras Gyori
> Priority: Major
>
> Mostly due to CapacitySchedulerConfiguration#getPropsWithPrefix or similar
> methods, the CapacityScheduler#reinit method has some quadratic complexity
> part with respect to queue numbers. Over 1000+ queues, it is a matter of
> minutes, which is too high to be a viable option when it is used in mutation
> api.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
