[
https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xianyin Xin updated YARN-4090:
------------------------------
Attachment: sampling2.jpg
sampling1.jpg
I construct a queue hierarchy with 3 levels,
root
child1 child2 child3
child1.child1~10, child2.child1~15, child3.child1~15
the number of leaf queues is 40. A total of 1000 apps running randomly on the
leaf queues. The sampling results show that about 2/3 of the cpu times of
FSParentQueue.assignContainers() was spent on Collections.sort(). In
Collections.sort(), about 40% was spent on
SchedulerAppplicationAttempt.getCurrentConsumption() and about 36% was spent on
Resources.substract(). The former time consuming is because
FSParentQueue.getResourceUsage() will make recursion on it's children, while
for the latter time consuming, the clone() in substract() takes much cpu time.
> Make Collections.sort() more efficient in FSParentQueue.java
> ------------------------------------------------------------
>
> Key: YARN-4090
> URL: https://issues.apache.org/jira/browse/YARN-4090
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Reporter: Xianyin Xin
> Attachments: sampling1.jpg, sampling2.jpg
>
>
> Collections.sort() consumes too much time in a scheduling round.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
