[
https://issues.apache.org/jira/browse/MESOS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153688#comment-15153688
]
Klaus Ma commented on MESOS-3283:
---------------------------------
[~bmahler], enhance the performance of addSlave (1000 slave and 100 frameworks)
from ~25s to ~4s:
1. If available resources is not allocatable, did not continue to check other
frameworks (~25s to ~11s)
2. Only update modified framework instead of sorting frameworks in each loop
(~11s to ~4)
Post a draft diff at https://reviews.apache.org/r/43754/ , I'm working to check
other case in allocator.
> Improve allocation performance especially with large number of slaves and
> frameworks.
> -------------------------------------------------------------------------------------
>
> Key: MESOS-3283
> URL: https://issues.apache.org/jira/browse/MESOS-3283
> Project: Mesos
> Issue Type: Improvement
> Components: allocation
> Affects Versions: 0.23.0
> Reporter: Mandeep Chadha
> Assignee: Klaus Ma
> Labels: mesosphere, tech-debt
>
> Improve batch allocations performance especially with large number of slaves
> and frameworks.
> e.g. these are the allocation timings for 10K slaves and varying number of
> frameworks.
> Using 10000 slaves and 1 frameworks
> Added 10000 slaves in 14.50836112secs
> Updated 10000 slaves in 18.665093703secs
> [ OK ]
> SlaveCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/12 (34983
> ms)
> [ RUN ]
> SlaveCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/13
> Using 10000 slaves and 50 frameworks
> Added 10000 slaves in 51.534229549secs
> Updated 10000 slaves in 57.131554303secs
> [ OK ]
> SlaveCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/13 (110449
> ms)
> [ RUN ]
> SlaveCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/14
> Using 10000 slaves and 100 frameworks
> Added 10000 slaves in 1.5891310434mins
> Updated 10000 slaves in 1.80562078148333mins
> [ OK ]
> SlaveCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/14 (205467
> ms)
> [ RUN ]
> SlaveCount/HierarchicalAllocator_BENCHMARK_Test.AddAndUpdateSlave/15
> Using 10000 slaves and 200 frameworks
> Added 10000 slaves in 3.0750647275mins
> Updated 10000 slaves in 3.85846762096667mins
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
