> On May 24, 2017, 11:35 p.m., James Peach wrote: > > src/tests/hierarchical_allocator_tests.cpp > > Lines 4972 (patched) > > <https://reviews.apache.org/r/49571/diff/35/?file=1730051#file1730051line4972> > > > > Why do we need to loop that many times? I don't think you'd expect the > > performance to vary much over this range because tou are always recovering > > the resources so the allocation state doesn't change substantially.
I now run the loop 6 times to ensure we cover each framework which should be `max(number_of_frameworks)/min(number_of_agents)`, i.e. `6000/1000 = 6`. > On May 24, 2017, 11:35 p.m., James Peach wrote: > > src/tests/hierarchical_allocator_tests.cpp > > Lines 4995 (patched) > > <https://reviews.apache.org/r/49571/diff/35/?file=1730051#file1730051line4995> > > > > I found that this output wasn't very helpful. How about running through > > the offer cycle N times and collecting the results in a > > `process::TimeSeries`, then showing the `process::Statistics` once at the > > end of the test? I think we should defer this to a future commit to do this for all benchmarks. - Anindya ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49571/#review176019 ----------------------------------------------------------- On May 25, 2017, 11:30 p.m., Anindya Sinha wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49571/ > ----------------------------------------------------------- > > (Updated May 25, 2017, 11:30 p.m.) > > > Review request for mesos, James Peach and Jiang Yan Xu. > > > Bugs: MESOS-5771 > https://issues.apache.org/jira/browse/MESOS-5771 > > > Repository: mesos > > > Description > ------- > > Allocations test has the following resource configurations: > (1) REGULAR: Offers from every slave have regular resources. > (2) SHARED: Offers from every slave include a shared resource. > (3) REGULAR: Offers from every alternate slave contain only regular > resources; and offers from every other alternate slave contains > a shared resource. > > This test is parameterized based on number of agents, number of > frameworks and resource configuration. > > > Diffs > ----- > > src/tests/hierarchical_allocator_tests.cpp > eb2b647d3247a85e8b9b82e5589232c74ad8570f > src/tests/resources_utils.hpp 1f41f02babce5c8174ea2223f4dc7470452fbaf1 > src/tests/resources_utils.cpp 2cef55f7312d671307e097c2c4960c8dcf45c1ff > > > Diff: https://reviews.apache.org/r/49571/diff/36/ > > > Testing > ------- > > All tests passed. > > Allocations benchmark test results > ================================== > There is no visible impact in performance when shared resources are added in > the allocations. The numbers for HEAD are prior to shared resources support > (mid 2016) and the numbers indicate improvements in allocations during this > timeframe. > > Following is a snapshot with 1000 agents and 200 frameworks. > > With the patch (and no shared resources) > ---------------------------------------- > [ RUN ] AllResources/HierarchicalAllocations_BENCHMARK_Test.Allocations/9 > Using 1000 agents and 200 frameworks with resource type 0 > Added 200 frameworks in 6588us > Added 1000 agents in 1.567347secs > round 0 allocate() took 1.15531secs to make 1000 offers > round 10 allocate() took 1.152876secs to make 1000 offers > round 20 allocate() took 1.15661secs to make 1000 offers > round 30 allocate() took 1.117733secs to make 1000 offers > round 40 allocate() took 1.118754secs to make 1000 offers > round 50 allocate() took 1.11169secs to make 1000 offers > > With the patch (and shared resources on all agents) > --------------------------------------------------- > [ RUN ] > AllResources/HierarchicalAllocations_BENCHMARK_Test.Allocations/10 > Using 1000 agents and 200 frameworks with resource type 1 > Added 200 frameworks in 6064us > Added 1000 agents in 1.627008secs > round 0 allocate() took 1.168253secs to make 1000 offers > round 10 allocate() took 1.146421secs to make 1000 offers > round 20 allocate() took 1.16416secs to make 1000 offers > round 30 allocate() took 1.210476secs to make 1000 offers > round 40 allocate() took 1.194251secs to make 1000 offers > round 50 allocate() took 1.17789secs to make 1000 offers > > With the patch (and shared resources on alternate agents) > --------------------------------------------------------- > [ RUN ] > AllResources/HierarchicalAllocations_BENCHMARK_Test.Allocations/11 > Using 1000 agents and 200 frameworks with resource type 2 > Added 200 frameworks in 6466us > Added 1000 agents in 1.568717secs > round 0 allocate() took 1.153005secs to make 1000 offers > round 10 allocate() took 1.168169secs to make 1000 offers > round 20 allocate() took 1.156774secs to make 1000 offers > round 30 allocate() took 1.183112secs to make 1000 offers > round 40 allocate() took 1.202452secs to make 1000 offers > round 50 allocate() took 1.198918secs to make 1000 offers > > Based on HEAD, with all regular resources (no shared resources in HEAD > supported) > --------------------------------------------------------------------------------- > [ RUN ] AllResources/HierarchicalAllocations_BENCHMARK_Test.Allocations/9 > Using 1000 agents and 200 frameworks with resource type 0 > Added 200 frameworks in 6801us > Added 1000 agents in 1.721447secs > round 0 allocate() took 1.502953secs to make 1000 offers > round 50 allocate() took 1.520157secs to make 1000 offers > round 100 allocate() took 1.517221secs to make 1000 offers > round 150 allocate() took 1.526446secs to make 1000 offers > round 199 allocate() took 1.538005secs to make 1000 offers > > > Thanks, > > Anindya Sinha > >
