----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33689/#review82916 -----------------------------------------------------------
Looks good overall. A few suggestions below. src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java <https://reviews.apache.org/r/33689/#comment133768> Delete TODO. src/main/java/org/apache/aurora/scheduler/SchedulerModule.java <https://reviews.apache.org/r/33689/#comment133746> @Positive src/main/java/org/apache/aurora/scheduler/SchedulerModule.java <https://reviews.apache.org/r/33689/#comment133748> Suggest rephrasing adding a notion that this is the argument controlling interruption rate of otherwise blocking wait calls. src/main/java/org/apache/aurora/scheduler/SchedulerModule.java <https://reviews.apache.org/r/33689/#comment133749> Suggest wrapping these into a UserTaskLauncherSettings as a more scalable/lower overhead solution. See example here: https://github.com/apache/aurora/blob/ef0975655c04f0c2f3ecb6599d4e4beb9547f091/src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java#L259 src/main/java/org/apache/aurora/scheduler/SchedulerModule.java <https://reviews.apache.org/r/33689/#comment133762> Drop, there is already a binding like that above. src/main/java/org/apache/aurora/scheduler/UserTaskLauncher.java <https://reviews.apache.org/r/33689/#comment133753> Just use "maxBatchSize - updates.size()" to avoid second guessing "-1" origin. src/test/java/org/apache/aurora/scheduler/UserTaskLauncherTest.java <https://reviews.apache.org/r/33689/#comment133771> Use 0L instead to avoid any delays in unit tests. src/test/java/org/apache/aurora/scheduler/UserTaskLauncherTest.java <https://reviews.apache.org/r/33689/#comment133773> .once() is redundant src/test/java/org/apache/aurora/scheduler/UserTaskLauncherTest.java <https://reviews.apache.org/r/33689/#comment133778> s/Object/Void src/test/java/org/apache/aurora/scheduler/UserTaskLauncherTest.java <https://reviews.apache.org/r/33689/#comment133775> Suggest using 5 seconds as it's more than enough time to run a single update. - Maxim Khutornenko On May 7, 2015, 12:27 a.m., Ben Mahler wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33689/ > ----------------------------------------------------------- > > (Updated May 7, 2015, 12:27 a.m.) > > > Review request for Aurora, Maxim Khutornenko and Bill Farner. > > > Bugs: AURORA-1228 > https://issues.apache.org/jira/browse/AURORA-1228 > > > Repository: aurora > > > Description > ------- > > Now the processing of status updates is done asynchronously with batching to > insulate throughput from the expensive storage resource. Updates are placed > into a queue and consumed by another thread. If many updates arrive while > we're storing a batch of updates, these will be processed together in batch > rather than individually. > > > Diffs > ----- > > src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java > 7bb64dd913f0fe2fede95d50a061043dbb794ab4 > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java > 45de15a57baf7a2f7d437b590935714e28777f35 > src/main/java/org/apache/aurora/scheduler/SchedulerModule.java > d3ac176e9402a33fd2074b0737313458120da9e2 > src/main/java/org/apache/aurora/scheduler/UserTaskLauncher.java > 0ce9c9d4cf75f9add260f285115b1d60786ded57 > src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java > 4d589a33a2933b0cb6caf85abfae45c5e635c3ce > src/main/java/org/apache/aurora/scheduler/mesos/Driver.java > c7e45a89ceaa2c310feb610091eec0b04187860e > src/main/java/org/apache/aurora/scheduler/mesos/MesosSchedulerImpl.java > 9b8ab7c1027731f9d3f6cae77b85272ea63354d4 > src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java > da2d5df2e053e6e1b8fb08d6813dff9eac9777f8 > src/test/java/org/apache/aurora/scheduler/UserTaskLauncherTest.java > 32432322753799562d671db39c0d7fa308d962ff > src/test/java/org/apache/aurora/scheduler/async/GcExecutorLauncherTest.java > 422d5a9a42310979752eb7282658316c2b772419 > src/test/java/org/apache/aurora/scheduler/mesos/MesosSchedulerImplTest.java > abdeee49858fc439c27911c4eb544bf8e8c931d4 > > Diff: https://reviews.apache.org/r/33689/diff/ > > > Testing > ------- > > Ran the benchmark to confirm that this improves status update throughput > substantially: > > Before: Around 100 updates per second for a 5ms storage latency. Much worse > for higher latencies. > After: Around 4k-5k updates per second for a 5ms storage latency, down to 3k > updates per second for 100ms storage latency. > > Updated unit tests for the new invariants: > > * TaskLaunchers are responsible for acknowledging updates. > * UserTaskLauncher processes updates asynchronously. > > > Thanks, > > Ben Mahler > >