> On June 27, 2018, 6:29 p.m., Benjamin Mahler wrote: > > Great to see this! > > > > Can you update the testing done to say whether this was measured with an > > optimized build? If not, can you add the optimized build measurements?
Updated numbers with optimized build > On June 27, 2018, 6:29 p.m., Benjamin Mahler wrote: > > src/tests/resources_tests.cpp > > Lines 3944 (patched) > > <https://reviews.apache.org/r/67590/diff/1/?file=2040451#file2040451line3963> > > > > The default port ranges in mesos is 31000-32000, and a user could set > > it to something like 10000-32000, so we probably also want to test in the > > tens of thousands worst case. > > > > E.g. 100, 1000, 10000 Updated the number. > On June 27, 2018, 6:29 p.m., Benjamin Mahler wrote: > > src/tests/resources_tests.cpp > > Lines 3947 (patched) > > <https://reviews.apache.org/r/67590/diff/1/?file=2040451#file2040451line3966> > > > > After figuring out what ranges are used, it looks like we probably want > > to specialize this name? E.g. ArithmeticOverlapping Done. > On June 27, 2018, 6:29 p.m., Benjamin Mahler wrote: > > src/tests/resources_tests.cpp > > Lines 3949-3970 (patched) > > <https://reviews.apache.org/r/67590/diff/1/?file=2040451#file2040451line3968> > > > > It's hard to understand what this is doing, can you describe what ports > > 1 and 2 look like or what the idea is here? Am I reading right? > > > > ports1 = [1-6, 11-16, 21-26, ...] > > ports2 = [3-8, 13-18, 23-28, ...] > > > > It looks like there's a lot of overlap in the ranges? It's probably > > worth describing why we do this (is it trying to match total-allocated?). > > We probably also want to name the test with "overlap" to clarify the > > relationship between port ranges in this test (vs say, ranges with no > > overlap). Done. The motivation for this benchmark was to find out the performance difference between addition and subtraction. The partial overlapping pattern represents a worst-case that is comparable for both addition and subtraction. It sounds like a good idea to test `total - allocated` which is probably the most common use case. I think it will be like one single big interval subtracts many small (allocated) intervals, though it no longer make sense to compare addition and subtraction. > On June 27, 2018, 6:29 p.m., Benjamin Mahler wrote: > > src/tests/resources_tests.cpp > > Lines 3954-3959 (patched) > > <https://reviews.apache.org/r/67590/diff/1/?file=2040451#file2040451line3973> > > > > Looks like this can be: > > > > ``` > > ports1 += createPorts(createRange(port1Index, port1Index + stride); > > ``` > > > > > > https://github.com/apache/mesos/blob/1.6.0/src/tests/resources_utils.hpp#L45-L62 Thanks. `createPorts` takes in ranges. But made similar improvements on the readability. - Meng ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67590/#review205488 ----------------------------------------------------------- On June 28, 2018, 12:34 p.m., Meng Zhu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/67590/ > ----------------------------------------------------------- > > (Updated June 28, 2018, 12:34 p.m.) > > > Review request for mesos and Benjamin Mahler. > > > Bugs: MESOS-8989 > https://issues.apache.org/jira/browse/MESOS-8989 > > > Repository: mesos > > > Description > ------- > > See summary. > > Also removed the current range benchmark which could produce > misleading results for subtraction (See MESOS-8989). > > > Diffs > ----- > > src/tests/resources_tests.cpp 0095b945ebfd9be52162547f3999826c8cc33f87 > > > Diff: https://reviews.apache.org/r/67590/diff/2/ > > > Testing > ------- > > Build with `-O2` optimization, ran on a multicore machine with peak frequency > at 2.2GHz: > > Took 23.31231ms to perform 1000 'a += b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 100 > Took 85.494021ms to perform 1000 'a -= b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 100 > Took 45.536174ms to perform 1000 'a + b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 100 > Took 108.754885ms to perform 1000 'a - b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 100 > > Took 309.716888ms to perform 1000 'a += b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 1000 > Took 917.667222ms to perform 1000 'a -= b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 1000 > Took 502.702844ms to perform 1000 'a + b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 1000 > Took 1.178407509secs to perform 1000 'a - b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 1000 > > Took 3.144658342secs to perform 1000 'a += b' operations on ports:[1-6, > 11-16, 21-26... and ports:[3-8, 13-18, 23-28... with size 10000 > Took 9.086052071secs to perform 1000 'a -= b' operations on ports:[1-6, > 11-16, 21-26... and ports:[3-8, 13-18, 23-28... with size 10000 > Took 4.518318425secs to perform 1000 'a + b' operations on ports:[1-6, 11-16, > 21-26... and ports:[3-8, 13-18, 23-28... with size 10000 > Took 10.604580315secs to perform 1000 'a - b' operations on ports:[1-6, > 11-16, 21-26... and ports:[3-8, 13-18, 23-28... with size 10000 > > > Thanks, > > Meng Zhu > >
