----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55217/#review160830 -----------------------------------------------------------
Ship it! Ship It! - Reza Motamedi On Jan. 5, 2017, 6:59 p.m., Mehrdad Nurolahzade wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55217/ > ----------------------------------------------------------- > > (Updated Jan. 5, 2017, 6:59 p.m.) > > > Review request for Aurora, Joshua Cohen and Stephan Erb. > > > Bugs: AURORA-1847 > https://issues.apache.org/jira/browse/AURORA-1847 > > > Repository: aurora > > > Description > ------- > > If scheduler is configured to run with the `MemTaskStore` every hit on > scheduler landing page (`/scheduler`) causes a call to > `MemTaskStore.getJobKeys()` through `ReadOnlyScheduler.getRoleSummary()`. > > The implementation of `MemTaskStore.getJobKeys()` is currently very > inefficient as it requires a sequential scan of the task store and mapping to > their respective job keys. In Twitter clusters this method is currently > taking half a second per call (`mem_storage_get_job_keys`). > > This patch eliminates the sequential scan and mapping to job key by simply > returning an immutable copy of the key set of the existing secondary index > `job`. > > > Diffs > ----- > > src/jmh/java/org/apache/aurora/benchmark/TaskStoreBenchmarks.java > f2f00b92bf901c438e40bbc177e9f5a112be1bbc > src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java > fc272ddb45be8b2f55f01c3d54f7fa9058202c0b > > Diff: https://reviews.apache.org/r/55217/diff/ > > > Testing > ------- > > Using the following modified version of existing > `TaskStoreBenchmarks.MemFetchTasksBenchmark` which benchmarks > `TaskStore.getJobKeys()`: > ```java > public static class MemFetchTasksBenchmark extends > AbstractFetchTasksBenchmark { > @Setup(Level.Trial) > @Override > public void setUp() { > storage = Guice.createInjector( > Modules.combine( > DbModule.testModuleWithWorkQueue(PLAIN, Optional.of(new > InMemStoresModule(PLAIN))), > new AbstractModule() { > @Override > protected void configure() { > bind(StatsProvider.class).toInstance(new > FakeStatsProvider()); > bind(Clock.class).toInstance(new FakeClock()); > } > })) > .getInstance(Storage.class); > > } > > @Setup(Level.Iteration) > public void setUpIteration() { > createTasks(numTasks); > } > > @TearDown(Level.Iteration) > public void tearDownIteration() { > deleteTasks(); > } > > @Benchmark > public Iterable<IJobKey> run() { > return storage.read(store -> store.getTaskStore().getJobKeys()); > } > } > ``` > > Benchmark results BEFORE patch: > ``` > Benchmark (numTasks) Mode Cnt > Score Error Units > TaskStoreBenchmarks.MemFetchTasksBenchmark.run 10000 thrpt 5 > 572.761 ± 168.865 ops/s > TaskStoreBenchmarks.MemFetchTasksBenchmark.run 50000 thrpt 5 > 80.697 ± 8.516 ops/s > TaskStoreBenchmarks.MemFetchTasksBenchmark.run 100000 thrpt 5 > 25.102 ± 3.244 ops/s > ``` > > Benchmark results AFTER patch: > ``` > Benchmark (numTasks) Mode Cnt > Score Error Units > TaskStoreBenchmarks.MemFetchTasksBenchmark.run 10000 thrpt 5 > 327336.998 ± 10207.402 ops/s > TaskStoreBenchmarks.MemFetchTasksBenchmark.run 50000 thrpt 5 > 320506.958 ± 23430.527 ops/s > TaskStoreBenchmarks.MemFetchTasksBenchmark.run 100000 thrpt 5 > 287962.695 ± 51917.245 ops/s > ``` > > > Thanks, > > Mehrdad Nurolahzade > >
