> On July 14, 2016, 7:14 p.m., Benjamin Mahler wrote: > > src/tests/sorter_tests.cpp, lines 504-542 > > <https://reviews.apache.org/r/49843/diff/3/?file=1442032#file1442032line504> > > > > It looks like we can simplify these two functions down to the following > > single function? > > > > ``` > > // Returns a "ports" resource with the number of ranges > > // specified as: [1-2, 4-5, 7-8, 10-11, ...] > > static Resource makePortRanges(size_t numRanges) > > { > > ::mesos::Value::Ranges ranges; > > > > for (size_t i = 0; i < numRanges; ++i) { > > Value::Range* range = ranges.add_range(); > > range->set_begin((3 * i) + 1); > > range->set_end(range->begin() + 1); > > } > > > > Value value; > > value.set_type(Value::RANGES); > > value.mutable_ranges()->CopyFrom(ranges); > > > > Resource resource; > > resource.set_role("*"); > > resource.set_name("ports"); > > resource.set_type(Value::RANGES); > > resource.mutable_ranges()->CopyFrom(value.ranges()); > > > > return resource; > > } > > ``` > > > > Since we only care about the number of ranges, we don't need to specify > > the bounds, right? > > Guangya Liu wrote: > Yes, my orignial thinking was keep this same as > https://github.com/apache/mesos/blob/master/src/tests/hierarchical_allocator_tests.cpp#L85-L122 > in case we may need specify bounds in future, but since this is only for > test and we do not need to care for bounds, so seems no need to have > `makeRange` here. I was thinking we may also need to update > https://github.com/apache/mesos/blob/master/src/tests/hierarchical_allocator_tests.cpp#L85-L122 > as well to use a single function to create port ranges. > > Guangya Liu wrote: > Hi Ben, there is one issue for current function: We are now using port > range as `[1-2, 4-5, 7-8, 10-11, ...]`, but the slave created using port > `[31000-32000]` in here > https://github.com/apache/mesos/blob/master/src/tests/sorter_tests.cpp#L565-L566 > . > > Basically, clients cannot get port resources which are not in > `[31000-32000]` but here we are using port range `[1-2, 4-5, 7-8, 10-11, > ...]`, this will not impact the `sorter` result as the port resources are > excluded from sort > https://github.com/apache/mesos/blob/master/src/master/allocator/sorter/drf/sorter.cpp#L288 > . > > But I think we still need to fix this to make sure the logic tecnically > right here, there are two options: 1) Keep the orignial code and have two > functions by using `makeRange` to define the bounds. 2) Hard code the > resources in slave and make the port range starting from 1. I think 1) is > better as this can make the port range controllable via some parameters, and > for 2), we need to make sure the allocated port range always fall into the > agent resources port range when setting the agent resources. > > What do you think?
Now I see why there were two functions! Sorry about that. Let's call it something like `fragment(...)` to make it clearer that it is taking a range and fragmenting it into subranges. Let's also return a Try since there are input cases that cannot be satisfied. E.g. [1-10] can be fragmented into at most 5 ranges [1,3,5,7,9], so `fragment([1-10], 6)` returns an error. - Benjamin ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49843/#review142264 ----------------------------------------------------------- On July 12, 2016, 9:09 a.m., Guangya Liu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49843/ > ----------------------------------------------------------- > > (Updated July 12, 2016, 9:09 a.m.) > > > Review request for mesos, Benjamin Mahler and Klaus Ma. > > > Bugs: MESOS-5701 > https://issues.apache.org/jira/browse/MESOS-5701 > > > Repository: mesos > > > Description > ------- > > Added benchmark test for sorter. > > > Diffs > ----- > > src/tests/sorter_tests.cpp 20e42419934e81b97676569876cd63ee0a550da3 > > Diff: https://reviews.apache.org/r/49843/diff/ > > > Testing > ------- > > make > make check > > ./bin/mesos-tests.sh --benchmark > --gtest_filter="AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/*" > [==========] Running 36 tests from 1 test case. > [----------] Global test environment set-up. > [----------] 36 tests from AgentAndClientCount/Sorter_BENCHMARK_Test > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/0 > Using 1000 agents and 1 clients > Added 1 clients in 1047us > Added 1000 agents in 30147us > Sorted 1 clients in 87us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/0 (33 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/1 > Using 1000 agents and 50 clients > Added 50 clients in 884us > Added 1000 agents in 30129us > Sorted 50 clients in 1284us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/1 (37 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/2 > Using 1000 agents and 100 clients > Added 100 clients in 1676us > Added 1000 agents in 25157us > Sorted 100 clients in 3814us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/2 (41 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/3 > Using 1000 agents and 200 clients > Added 200 clients in 3930us > Added 1000 agents in 25743us > Sorted 200 clients in 7780us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/3 (58 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/4 > Using 1000 agents and 500 clients > Added 500 clients in 9219us > Added 1000 agents in 29179us > Sorted 500 clients in 16210us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/4 (112 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/5 > Using 1000 agents and 1000 clients > Added 1000 clients in 22029us > Added 1000 agents in 28043us > Sorted 1000 clients in 36030us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/5 (192 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/6 > Using 5000 agents and 1 clients > Added 1 clients in 43us > Added 5000 agents in 134324us > Sorted 1 clients in 42us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/6 (136 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/7 > Using 5000 agents and 50 clients > Added 50 clients in 845us > Added 5000 agents in 137171us > Sorted 50 clients in 1274us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/7 (144 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/8 > Using 5000 agents and 100 clients > Added 100 clients in 1698us > Added 5000 agents in 137456us > Sorted 100 clients in 2582us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/8 (152 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/9 > Using 5000 agents and 200 clients > Added 200 clients in 3460us > Added 5000 agents in 129285us > Sorted 200 clients in 5587us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/9 (159 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/10 > Using 5000 agents and 500 clients > Added 500 clients in 8871us > Added 5000 agents in 133412us > Sorted 500 clients in 16717us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/10 (212 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/11 > Using 5000 agents and 1000 clients > Added 1000 clients in 22705us > Added 5000 agents in 143366us > Sorted 1000 clients in 36587us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/11 (313 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/12 > Using 10000 agents and 1 clients > Added 1 clients in 34us > Added 10000 agents in 284603us > Sorted 1 clients in 43us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/12 (286 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/13 > Using 10000 agents and 50 clients > Added 50 clients in 857us > Added 10000 agents in 270253us > Sorted 50 clients in 1192us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/13 (279 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/14 > Using 10000 agents and 100 clients > Added 100 clients in 1786us > Added 10000 agents in 282553us > Sorted 100 clients in 2660us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/14 (297 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/15 > Using 10000 agents and 200 clients > Added 200 clients in 3447us > Added 10000 agents in 270059us > Sorted 200 clients in 6696us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/15 (302 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/16 > Using 10000 agents and 500 clients > Added 500 clients in 10724us > Added 10000 agents in 271856us > Sorted 500 clients in 16945us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/16 (353 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/17 > Using 10000 agents and 1000 clients > Added 1000 clients in 21394us > Added 10000 agents in 274222us > Sorted 1000 clients in 38006us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/17 (437 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/18 > Using 20000 agents and 1 clients > Added 1 clients in 44us > Added 20000 agents in 545506us > Sorted 1 clients in 50us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/18 (549 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/19 > Using 20000 agents and 50 clients > Added 50 clients in 823us > Added 20000 agents in 558720us > Sorted 50 clients in 1270us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/19 (568 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/20 > Using 20000 agents and 100 clients > Added 100 clients in 1682us > Added 20000 agents in 603786us > Sorted 100 clients in 2689us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/20 (623 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/21 > Using 20000 agents and 200 clients > Added 200 clients in 3563us > Added 20000 agents in 639230us > Sorted 200 clients in 7246us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/21 (681 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/22 > Using 20000 agents and 500 clients > Added 500 clients in 9577us > Added 20000 agents in 603457us > Sorted 500 clients in 16546us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/22 (691 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/23 > Using 20000 agents and 1000 clients > Added 1000 clients in 18104us > Added 20000 agents in 517677us > Sorted 1000 clients in 31077us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/23 (677 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/24 > Using 30000 agents and 1 clients > Added 1 clients in 47us > Added 30000 agents in 856802us > Sorted 1 clients in 46us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/24 (862 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/25 > Using 30000 agents and 50 clients > Added 50 clients in 1665us > Added 30000 agents in 883548us > Sorted 50 clients in 1259us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/25 (895 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/26 > Using 30000 agents and 100 clients > Added 100 clients in 1619us > Added 30000 agents in 834926us > Sorted 100 clients in 2630us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/26 (855 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/27 > Using 30000 agents and 200 clients > Added 200 clients in 3288us > Added 30000 agents in 798885us > Sorted 200 clients in 7605us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/27 (834 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/28 > Using 30000 agents and 500 clients > Added 500 clients in 8797us > Added 30000 agents in 798340us > Sorted 500 clients in 17014us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/28 (882 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/29 > Using 30000 agents and 1000 clients > Added 1000 clients in 20749us > Added 30000 agents in 817770us > Sorted 1000 clients in 35576us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/29 (976 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/30 > Using 50000 agents and 1 clients > Added 1 clients in 47us > Added 50000 agents in 1.502226secs > Sorted 1 clients in 56us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/30 (1512 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/31 > Using 50000 agents and 50 clients > Added 50 clients in 1150us > Added 50000 agents in 1.367002secs > Sorted 50 clients in 1820us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/31 (1385 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/32 > Using 50000 agents and 100 clients > Added 100 clients in 1812us > Added 50000 agents in 1.374993secs > Sorted 100 clients in 2552us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/32 (1395 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/33 > Using 50000 agents and 200 clients > Added 200 clients in 3197us > Added 50000 agents in 1.319963secs > Sorted 200 clients in 5672us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/33 (1356 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/34 > Using 50000 agents and 500 clients > Added 500 clients in 8942us > Added 50000 agents in 1.368761secs > Sorted 500 clients in 14604us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/34 (1459 ms) > [ RUN ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/35 > Using 50000 agents and 1000 clients > Added 1000 clients in 19147us > Added 50000 agents in 1.401624secs > Sorted 1000 clients in 35276us > [ OK ] AgentAndClientCount/Sorter_BENCHMARK_Test.FullSort/35 (1579 ms) > [----------] 36 tests from AgentAndClientCount/Sorter_BENCHMARK_Test (21323 > ms total) > > [----------] Global test environment tear-down > [==========] 36 tests from 1 test case ran. (21345 ms total) > [ PASSED ] 36 tests. > > > Thanks, > > Guangya Liu > >
