Re: Review Request 28474: Added manual perf tests for the scheduling pipeline.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28474/#review63888
---
Discarding this in favor of benchmark harness started in
https://reviews.apache.org/r/28710/.
- Maxim Khutornenko
On Nov. 26, 2014, 6:15 p.m., Maxim Khutornenko wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28474/
---
(Updated Nov. 26, 2014, 6:15 p.m.)
Review request for Aurora and Bill Farner.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description
---
A manual testbed for the upcoming AURORA-909 work.
Diffs
-
src/test/java/org/apache/aurora/scheduler/async/Offers.java
8293dd181b0d062e89776fdc1205c1c227d6bb6c
src/test/java/org/apache/aurora/scheduler/async/SchedulerPerfIT.java
PRE-CREATION
Diff: https://reviews.apache.org/r/28474/diff/
Testing
---
./gradlew -Pq build
IDE manual run:
Nov 26, 2014 10:12:21 AM
org.apache.aurora.scheduler.async.preemptor.PreemptorModule configure
INFO: Preemptor Enabled.
Nov 26, 2014 10:12:21 AM com.twitter.common.util.BuildInfo fetchProperties
INFO: Fetching build properties from build.properties
Nov 26, 2014 10:12:21 AM com.twitter.common.util.BuildInfo fetchProperties
WARNING: Failed to fetch build properties from build.properties
Nov 26, 2014 10:12:21 AM
com.twitter.common.application.modules.StatsModule$StartStatPoller execute
INFO: Build information: {}
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
runTest
INFO: Results for: INSUFFICIENT_RESOURCES
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
logDuration
INFO: Mean task_schedule_attempt duration: 737660.0796871068 ns
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
logDuration
INFO: Mean offer_queue_launch_first duration: 332330.25185686833 ns
Thanks,
Maxim Khutornenko
Re: Review Request 28474: Added manual perf tests for the scheduling pipeline.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28474/#review63119
---
Ship it!
Master (ae4d70f) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing @ReviewBot
retry
- Aurora ReviewBot
On Nov. 26, 2014, 6:15 p.m., Maxim Khutornenko wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28474/
---
(Updated Nov. 26, 2014, 6:15 p.m.)
Review request for Aurora and Bill Farner.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description
---
A manual testbed for the upcoming AURORA-909 work.
Diffs
-
src/test/java/org/apache/aurora/scheduler/async/Offers.java
8293dd181b0d062e89776fdc1205c1c227d6bb6c
src/test/java/org/apache/aurora/scheduler/async/SchedulerPerfIT.java
PRE-CREATION
Diff: https://reviews.apache.org/r/28474/diff/
Testing
---
./gradlew -Pq build
IDE manual run:
Nov 26, 2014 10:12:21 AM
org.apache.aurora.scheduler.async.preemptor.PreemptorModule configure
INFO: Preemptor Enabled.
Nov 26, 2014 10:12:21 AM com.twitter.common.util.BuildInfo fetchProperties
INFO: Fetching build properties from build.properties
Nov 26, 2014 10:12:21 AM com.twitter.common.util.BuildInfo fetchProperties
WARNING: Failed to fetch build properties from build.properties
Nov 26, 2014 10:12:21 AM
com.twitter.common.application.modules.StatsModule$StartStatPoller execute
INFO: Build information: {}
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
runTest
INFO: Results for: INSUFFICIENT_RESOURCES
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
logDuration
INFO: Mean task_schedule_attempt duration: 737660.0796871068 ns
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
logDuration
INFO: Mean offer_queue_launch_first duration: 332330.25185686833 ns
Thanks,
Maxim Khutornenko
Re: Review Request 28474: Added manual perf tests for the scheduling pipeline.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28474/#review63130
---
Thanks a lot for breaking ground on this! I've been thinking about this for a
while, and i feel there's some additional structure needed to get it right:
a. for timing information to be valuable, we really need a high number of
iterations (leading to b.). Azul posted a great presentation [1] on java
benchmarking gotchas that i highly recommend to all members of the team (i'm
going to re-read it, myself). There's a similar two-part IBM article [2] that
also includes sample code and more statistical analysis.
b. we need to look into existing microbenchmark frameworks, such as caliper [3]
(gradle example [4]), and jmh [5] (plus jmh-gradle-plugin [6]), and others i'm
sure.
c. a benchmark producing useful data will likely be too long-running to include
in our normal unit tests, so some gradle work will be needed to split them out.
Perhaps we capture '*Benchmark.java' as a separate gradle test task? See [7]
and section 23.7.3 in [8] for examples.
d. the suite we put together here would greatly benefit from different input
data sets to serve as performance smoke tests - for example, a very large task
group should not starve small task groups, multiple large task groups should be
~equally served, and high numbers of task groups. we don't need to figure out
all of these right now, but building the scaffolding this will be really useful
for building future data sets to serve as regression tests
e. more a gut instinct than anything else, but we should probably use fakes
instead of mocks in performance tests to (hopefully) make them easier to write,
but also to avoid unexpected synchronization tainting timing measurements
[1] http://www.azulsystems.com/presentations/art-of-java-benchmarking
[2] http://www.ibm.com/developerworks/library/j-benchmark1/index.html
[3] https://code.google.com/p/caliper/
[4]
http://stackoverflow.com/questions/10099804/defining-a-gradle-task-to-run-caliper-microbenchmark
[5] http://openjdk.java.net/projects/code-tools/jmh/
[6] https://github.com/melix/jmh-gradle-plugin
[7]
http://forums.gradle.org/gradle/topics/gradle_does_not_support_the_old_way_of_defining_junit_test_suites
[8] http://www.gradle.org/docs/current/userguide/java_plugin.html
- Bill Farner
On Nov. 26, 2014, 6:15 p.m., Maxim Khutornenko wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28474/
---
(Updated Nov. 26, 2014, 6:15 p.m.)
Review request for Aurora and Bill Farner.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description
---
A manual testbed for the upcoming AURORA-909 work.
Diffs
-
src/test/java/org/apache/aurora/scheduler/async/Offers.java
8293dd181b0d062e89776fdc1205c1c227d6bb6c
src/test/java/org/apache/aurora/scheduler/async/SchedulerPerfIT.java
PRE-CREATION
Diff: https://reviews.apache.org/r/28474/diff/
Testing
---
./gradlew -Pq build
IDE manual run:
Nov 26, 2014 10:12:21 AM
org.apache.aurora.scheduler.async.preemptor.PreemptorModule configure
INFO: Preemptor Enabled.
Nov 26, 2014 10:12:21 AM com.twitter.common.util.BuildInfo fetchProperties
INFO: Fetching build properties from build.properties
Nov 26, 2014 10:12:21 AM com.twitter.common.util.BuildInfo fetchProperties
WARNING: Failed to fetch build properties from build.properties
Nov 26, 2014 10:12:21 AM
com.twitter.common.application.modules.StatsModule$StartStatPoller execute
INFO: Build information: {}
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
runTest
INFO: Results for: INSUFFICIENT_RESOURCES
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
logDuration
INFO: Mean task_schedule_attempt duration: 737660.0796871068 ns
Nov 26, 2014 10:12:26 AM org.apache.aurora.scheduler.async.SchedulerPerfIT
logDuration
INFO: Mean offer_queue_launch_first duration: 332330.25185686833 ns
Thanks,
Maxim Khutornenko
