> On Nov. 23, 2016, 4:36 a.m., Stephan Erb wrote: > > Have you considerd writing a single benchmark with a single `run()` method > > that: > > > > * inserts pending tasks > > * assigns the pending tasks > > * deletes the tasks > > > > Even though the benchmark would not be very fine grained, I doubt that we > > would lose much precision because of the significant code sharing between > > all methods. In addtion, the benchmark would be easy to follow and would > > not require any setup or cleanup methods. > > > > What do you think?
The problem with this approach is that it becomes dominated by task assignment since you can only assign one task at a time but insert several thousand tasks or delete several thousand tasks. I picked these benchmarks because they clearly show improvements that I can make to insertions and deletions. If you insist, I can create this benchmark, but I would make it an adiditional one because these benchmarks allow me to show improvements to the bulk APIs. > On Nov. 23, 2016, 4:36 a.m., Stephan Erb wrote: > > src/jmh/java/org/apache/aurora/benchmark/StateManagerBenchmarks.java, line > > 82 > > <https://reviews.apache.org/r/54011/diff/1/?file=1568932#file1568932line82> > > > > Just to check my understanding: This will be the in memory task store, > > correct? Yes - Zameer ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54011/#review156727 ----------------------------------------------------------- On Nov. 22, 2016, 6:47 p.m., Zameer Manji wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54011/ > ----------------------------------------------------------- > > (Updated Nov. 22, 2016, 6:47 p.m.) > > > Review request for Aurora, Joshua Cohen and Stephan Erb. > > > Bugs: AURORA-1823 > https://issues.apache.org/jira/browse/AURORA-1823 > > > Repository: aurora > > > Description > ------- > > `StateManagerImpl` is in the middle of every task state transition in the > scheduler. Performance improvements here could yield scheduling throughput > improvements across the board. This adds benchmarks for the two bulk APIs, > inserting pending tasks and deleting tasks. Sample output: > > ```` > Benchmark (numPendingTasks) > (numTasksToDelete) Mode Cnt Score Error Units > StateManagerBenchmarks.DeleteTasksBenchmark.run N/A > 1000 thrpt 10 2.510 ± 0.557 ops/s > StateManagerBenchmarks.DeleteTasksBenchmark.run N/A > 10000 thrpt 10 0.272 ± 0.030 ops/s > StateManagerBenchmarks.DeleteTasksBenchmark.run N/A > 50000 thrpt 10 0.053 ± 0.011 ops/s > StateManagerBenchmarks.InsertPendingTasksBenchmark.run 1000 > N/A thrpt 10 2.446 ± 0.698 ops/s > StateManagerBenchmarks.InsertPendingTasksBenchmark.run 10000 > N/A thrpt 10 0.246 ± 0.018 ops/s > StateManagerBenchmarks.InsertPendingTasksBenchmark.run 50000 > N/A thrpt 10 0.041 ± 0.006 ops/s > ```` > > > Diffs > ----- > > src/jmh/java/org/apache/aurora/benchmark/StateManagerBenchmarks.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/54011/diff/ > > > Testing > ------- > > > Thanks, > > Zameer Manji > >
