----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49843/#review141779 -----------------------------------------------------------
src/tests/sorter_tests.cpp (lines 493 - 498) <https://reviews.apache.org/r/49843/#comment207145> Recall that the sorter has "clients", not "frameworks". Can you update the terminology? This is confusing for the reader. The comment here says that this parameterizes the number of frameworks but it actually also represents the number of agents. Also, the output you showed in the Testing Done section doesn't mention how many agents are used, which is pretty important to know. Can you also parameterize this seperately on the number of agents? The allocator benchmarks should have some example numbers: ``` ::testing::Values(1000U, 5000U, 10000U, 20000U, 30000U, 50000U) ``` For clients, let's copy the numbers from the allocator benchmarks as well? ``` ::testing::Values(1U, 50U, 100U, 200U, 500U, 1000U)) ``` src/tests/sorter_tests.cpp (lines 501 - 534) <https://reviews.apache.org/r/49843/#comment207149> We should more explicitly describe what we're looking to measure. It seems to me that you're trying to measure the time it takes to do a full sort from the dirty state? Perhaps it would be more straightforward to do this benchmark in steps: Step 1. Add S slaves. How long does this take? Step 2. Add C clients. How long does this take? Step 3. Fill the cluster with allocations. To make this simple to start with, you can allocate each agent's resources to single client (round robin through the clients). Step 4. Do a full sort. How long does this take? That should be a pretty comprehensive benchmark. We can call this Sorter_BENCHMARK_Test.FullSort. src/tests/sorter_tests.cpp (lines 509 - 510) <https://reviews.apache.org/r/49843/#comment207152> Let's also have "disk", "ports" here to try to get closer to a real world scenario. We don't need to do any complicated port fragmentation for now. - Benjamin Mahler On July 10, 2016, 10:27 p.m., Guangya Liu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49843/ > ----------------------------------------------------------- > > (Updated July 10, 2016, 10:27 p.m.) > > > Review request for mesos, Benjamin Mahler and Klaus Ma. > > > Bugs: MESOS-5701 > https://issues.apache.org/jira/browse/MESOS-5701 > > > Repository: mesos > > > Description > ------- > > Added benchmark test for sorter. > > > Diffs > ----- > > src/tests/sorter_tests.cpp 0a921347586808863e615ca3dcc23fae92b629f5 > > Diff: https://reviews.apache.org/r/49843/diff/ > > > Testing > ------- > > make > make check > > ./bin/mesos-tests.sh --benchmark > --gtest_filter="FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks*" > [==========] Running 4 tests from 1 test case. > [----------] Global test environment set-up. > [----------] 4 tests from FrameworkCount/Sorter_BENCHMARK_Test > [ RUN ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/0 > Sorted 1000 frameworks in 5443us > [ OK ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/0 (67 ms) > [ RUN ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/1 > Sorted 2000 frameworks in 10393us > [ OK ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/1 (114 ms) > [ RUN ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/2 > Sorted 3000 frameworks in 22161us > [ OK ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/2 (190 ms) > [ RUN ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/3 > Sorted 5000 frameworks in 31893us > [ OK ] FrameworkCount/Sorter_BENCHMARK_Test.SortFrameworks/3 (341 ms) > [----------] 4 tests from FrameworkCount/Sorter_BENCHMARK_Test (713 ms total) > > [----------] Global test environment tear-down > [==========] 4 tests from 1 test case ran. (728 ms total) > [ PASSED ] 4 tests. > > > Thanks, > > Guangya Liu > >
