----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55874/#review163611 -----------------------------------------------------------
Ship it! Ship It! - Jacob Janco On Jan. 24, 2017, 9:46 p.m., Jiang Yan Xu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55874/ > ----------------------------------------------------------- > > (Updated Jan. 24, 2017, 9:46 p.m.) > > > Review request for mesos, Benjamin Mahler, Jacob Janco, and Zhitao Li. > > > Bugs: MESOS-6904 > https://issues.apache.org/jira/browse/MESOS-6904 > > > Repository: mesos > > > Description > ------- > > To evaluate MESOS-6904. > > > Diffs > ----- > > src/tests/hierarchical_allocator_tests.cpp > 1edd0ecc8a93cd41532e1cf3641f67c780ab23a5 > > Diff: https://reviews.apache.org/r/55874/diff/ > > > Testing > ------- > > ## Interpretation the benchmark output > > - With the allocation batching it's faster to process all the `reviveOffers` > events after all resources in the cluster have already been allocated. These > allocation runs then don't generate allocations and so their execution should > be identical and the allocation batching reduces the number of them. > - With the allocation batching it takes about the same time to process the > same number of `addSlave` events, no matter if allocations are batched or > not. This is because individual `addSlave` calls trigger only allocation for > that agent while batched allocation allocates all of the accumulated > candidates so far, so the total cost should be the similar. > - It's also faster to process `addFrameworks` with allocation batching but in > this benchmark frameworks are added when no agents are connected, so the > allocation runs should exit very quickly. > > ## Output > > With the patch set: > > ``` > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/0 > Using 1000 agents and 1 frameworks > Added 1 frameworks in 1.638795ms with 1 allocation runs > Added 1000 agents in 799.084178ms with 4 allocation runs > Processed 1 `reviveOffers` calls in 10.030412ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/0 > (909 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/1 > Using 1000 agents and 50 frameworks > Added 50 frameworks in 5.496104ms with 2 allocation runs > Added 1000 agents in 1.210938725secs with 3 allocation runs > Processed 50 `reviveOffers` calls in 129.410729ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/1 > (1420 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/2 > Using 1000 agents and 100 frameworks > Added 100 frameworks in 9.31721ms with 2 allocation runs > Added 1000 agents in 1.064227156secs with 3 allocation runs > Processed 100 `reviveOffers` calls in 229.829276ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/2 > (1410 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/3 > Using 1000 agents and 200 frameworks > Added 200 frameworks in 19.380803ms with 3 allocation runs > Added 1000 agents in 1.472364101secs with 2 allocation runs > Processed 200 `reviveOffers` calls in 419.054445ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/3 > (2007 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/4 > Using 1000 agents and 500 frameworks > Added 500 frameworks in 39.074317ms with 2 allocation runs > Added 1000 agents in 2.561990683secs with 2 allocation runs > Processed 500 `reviveOffers` calls in 754.971627ms with 2 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/4 > (3440 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/5 > Using 1000 agents and 1000 frameworks > Added 1000 frameworks in 73.831892ms with 2 allocation runs > Added 1000 agents in 4.395376381secs with 2 allocation runs > Processed 1000 `reviveOffers` calls in 1.660467486secs with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/5 > (6218 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/6 > Using 1000 agents and 3000 frameworks > Added 3000 frameworks in 139.383796ms with 3 allocation runs > Added 1000 agents in 10.951388325secs with 2 allocation runs > Processed 3000 `reviveOffers` calls in 4.557919408secs with 2 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/6 > (15748 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/7 > Using 1000 agents and 6000 frameworks > Added 6000 frameworks in 316.791333ms with 3 allocation runs > Added 1000 agents in 21.610491338secs with 2 allocation runs > Processed 6000 `reviveOffers` calls in 8.869317036secs with 2 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/7 > (30921 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/8 > Using 5000 agents and 1 frameworks > Added 1 frameworks in 666231ns with 1 allocation runs > Added 5000 agents in 4.598545159secs with 4 allocation runs > Processed 1 `reviveOffers` calls in 47.113709ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/8 > (5003 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/9 > Using 5000 agents and 50 frameworks > Added 50 frameworks in 5.183658ms with 2 allocation runs > Added 5000 agents in 8.579621688secs with 3 allocation runs > Processed 50 `reviveOffers` calls in 661.889668ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/9 > (9588 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/10 > Using 5000 agents and 100 frameworks > Added 100 frameworks in 8.75169ms with 2 allocation runs > Added 5000 agents in 10.135994191secs with 3 allocation runs > Processed 100 `reviveOffers` calls in 958.393865ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/10 > (11439 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/11 > Using 5000 agents and 200 frameworks > Added 200 frameworks in 18.671253ms with 3 allocation runs > Added 5000 agents in 11.071903594secs with 3 allocation runs > Processed 200 `reviveOffers` calls in 1.712832633secs with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/11 > (13111 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/12 > Using 5000 agents and 500 frameworks > Added 500 frameworks in 39.538389ms with 3 allocation runs > Added 5000 agents in 16.969996362secs with 3 allocation runs > Processed 500 `reviveOffers` calls in 3.822943337secs with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/12 > (21204 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/13 > Using 5000 agents and 1000 frameworks > Added 1000 frameworks in 76.366173ms with 4 allocation runs > Added 5000 agents in 29.192506073secs with 3 allocation runs > Processed 1000 `reviveOffers` calls in 8.41620529secs with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/13 > (37994 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/14 > Using 5000 agents and 3000 frameworks > Added 3000 frameworks in 213.188218ms with 3 allocation runs > Added 5000 agents in 1.14503399673333mins with 2 allocation runs > Processed 3000 `reviveOffers` calls in 22.84809316secs with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/14 > (92087 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/15 > Using 5000 agents and 6000 frameworks > Added 6000 frameworks in 277.973128ms with 5 allocation runs > Added 5000 agents in 2.03803562768333mins with 3 allocation runs > Processed 6000 `reviveOffers` calls in 45.922090085secs with 2 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/15 > (168869 ms) > ... > ``` > > Without the patch set: > ``` > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/0 > Using 1000 agents and 1 frameworks > Added 1 frameworks in 1.851101ms with 1 allocation runs > Added 1000 agents in 658.956396ms with 1000 allocation runs > Processed 1 `reviveOffers` calls in 8.261096ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/0 > (760 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/1 > Using 1000 agents and 50 frameworks > Added 50 frameworks in 8.397789ms with 50 allocation runs > Added 1000 agents in 854.988219ms with 1000 allocation runs > Processed 50 `reviveOffers` calls in 233.817863ms with 50 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/1 > (1189 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/2 > Using 1000 agents and 100 frameworks > Added 100 frameworks in 14.322559ms with 100 allocation runs > Added 1000 agents in 1.056275232secs with 1000 allocation runs > Processed 100 `reviveOffers` calls in 431.223604ms with 100 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/2 > (1587 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/3 > Using 1000 agents and 200 frameworks > Added 200 frameworks in 32.182689ms with 200 allocation runs > Added 1000 agents in 1.433496448secs with 1000 allocation runs > Processed 200 `reviveOffers` calls in 833.051338ms with 200 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/3 > (2359 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/4 > Using 1000 agents and 500 frameworks > Added 500 frameworks in 72.557537ms with 500 allocation runs > Added 1000 agents in 2.762114651secs with 1000 allocation runs > Processed 500 `reviveOffers` calls in 2.050692828secs with 500 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/4 > (4968 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/5 > Using 1000 agents and 1000 frameworks > Added 1000 frameworks in 119.020379ms with 1000 allocation runs > Added 1000 agents in 4.329463959secs with 1000 allocation runs > Processed 1000 `reviveOffers` calls in 3.315772475secs with 1000 allocation > runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/5 > (7857 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/6 > Using 1000 agents and 3000 frameworks > Added 3000 frameworks in 307.91983ms with 3000 allocation runs > Added 1000 agents in 11.01207796secs with 1000 allocation runs > Processed 3000 `reviveOffers` calls in 9.572909873secs with 3000 allocation > runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/6 > (20991 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/7 > Using 1000 agents and 6000 frameworks > Added 6000 frameworks in 570.986147ms with 6000 allocation runs > Added 1000 agents in 21.43404308secs with 1000 allocation runs > Processed 6000 `reviveOffers` calls in 19.351272234secs with 6000 allocation > runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/7 > (41472 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/8 > Using 5000 agents and 1 frameworks > Added 1 frameworks in 474133ns with 1 allocation runs > Added 5000 agents in 2.520829265secs with 5000 allocation runs > Processed 1 `reviveOffers` calls in 24.411648ms with 1 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/8 > (2851 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/9 > Using 5000 agents and 50 frameworks > Added 50 frameworks in 6.965268ms with 50 allocation runs > Added 5000 agents in 3.646906559secs with 5000 allocation runs > Processed 50 `reviveOffers` calls in 1.000947584secs with 50 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/9 > (4948 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/10 > Using 5000 agents and 100 frameworks > Added 100 frameworks in 29.503655ms with 100 allocation runs > Added 5000 agents in 4.718859304secs with 5000 allocation runs > Processed 100 `reviveOffers` calls in 2.034795179secs with 100 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/10 > (7132 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/11 > Using 5000 agents and 200 frameworks > Added 200 frameworks in 31.761516ms with 200 allocation runs > Added 5000 agents in 6.71781219secs with 5000 allocation runs > Processed 200 `reviveOffers` calls in 3.682332737secs with 200 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/11 > (10742 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/12 > Using 5000 agents and 500 frameworks > Added 500 frameworks in 71.563168ms with 500 allocation runs > Added 5000 agents in 13.130200606secs with 5000 allocation runs > Processed 500 `reviveOffers` calls in 8.430420361secs with 500 allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/12 > (22008 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/13 > Using 5000 agents and 1000 frameworks > Added 1000 frameworks in 122.306748ms with 1000 allocation runs > Added 5000 agents in 32.502122389secs with 5000 allocation runs > Processed 1000 `reviveOffers` calls in 16.547209905secs with 1000 allocation > runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/13 > (49485 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/14 > Using 5000 agents and 3000 frameworks > Added 3000 frameworks in 301.394386ms with 3000 allocation runs > Added 5000 agents in 1.08586860435mins with 5000 allocation runs > Processed 3000 `reviveOffers` calls in 49.605255631secs with 3000 allocation > runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/14 > (115408 ms) > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/15 > Using 5000 agents and 6000 frameworks > Added 6000 frameworks in 563.918755ms with 6000 allocation runs > Added 5000 agents in 2.12307636961667mins with 5000 allocation runs > Processed 6000 `reviveOffers` calls in 1.68996078711667mins with 6000 > allocation runs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.AllocatorBacklog/15 > (229736 ms) > ... > ``` > > > Thanks, > > Jiang Yan Xu > >
