-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55284/#review160785
-----------------------------------------------------------
Master (d4ebb56) is red with this patch.
./build-support/jenkins/build.sh
:commons:processResources
:commons:classes
:commons:jar
:compileJava/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java:74:
Note: Wrote forwarder
org.apache.aurora.scheduler.storage.log.WriteAheadStorageForwarder
@Forward({
^
Note: Writing
file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/org/apache/aurora/common/args/apt/cmdline.arg.info.txt.2
Note: Writing
file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/META-INF/compiler/resource-mappings/org.apache.aurora.common.args.apt.CmdLineProcessor
:generateBuildProperties
:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJavaNote:
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh[ant:checkstyle] [ERROR]
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java:0:
File does not end with a newline. [NewlineAtEndOfFile]
[ant:checkstyle] [ERROR]
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java:1:
File contains a sequence of empty lines. [RegexpMultiline]
[ant:checkstyle] [ERROR]
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java:1:
Line does not match expected header line of '^\/\*\*$'. [RegexpHeader]
[ant:checkstyle] [ERROR]
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java:38:1:
Utility classes should not have a public or default constructor.
[HideUtilityClassConstructor]
FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkstyleJmh'.
> Checkstyle rule violations were found. See the report at:
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/jmh.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output.
BUILD FAILED
Total time: 1 mins 10.533 secs
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On Jan. 6, 2017, 11:11 p.m., Reza Motamedi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55284/
> -----------------------------------------------------------
>
> (Updated Jan. 6, 2017, 11:11 p.m.)
>
>
> Review request for Aurora, Joshua Cohen, Mehrdad Nurolahzade, and Santhosh
> Kumar Shanmugham.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> # Adding SLA benchmarks benchmarks
> This RB is related to
> [AURORA-1854](https://issues.apache.org/jira/browse/AURORA-1854).
> The goal is to add JMH benchmarks for SlaCalculation and then other metric
> calculators.
>
> I am not sure if the results are very resonable. For instance see that the
> ops/sec is 54.701±19.743, which seems to have a high variance.
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java
> PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java
> 5d8d5bd8f705770979f284d26d2e932aabe707e5
>
> Diff: https://reviews.apache.org/r/55284/diff/
>
>
> Testing
> -------
>
> Following is the benchmarks collectcted:
>
> # rmotamedi@tw-mbp-rmotamedi:~/workspace/aurora on git:jmh-sla ?? [14:39:56]
> ? ./gradlew jmh -Pbenchmarks='SlaAlgorithmBenchmark'
>
>
> the content of <./aurora/dist/reports/jmh/human.txt>
>
> ```
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 100)
>
> # Run progress: 0.00% complete, ETA 00:05:15
> # Fork: 1 of 1
> # Warmup Iteration 1: 127519.626 ops/s
> Iteration 1: 152258.072 ops/s
> Iteration 2: 170404.745 ops/s
> Iteration 3: 138346.324 ops/s
> Iteration 4: 150597.403 ops/s
> Iteration 5: 169640.531 ops/s
>
>
> Result "measureMTTA":
> 156249.415 ±(99.9%) 52660.004 ops/s [Average]
> (min, avg, max) = (138346.324, 156249.415, 170404.745), stdev = 13675.636
> CI (99.9%): [103589.411, 208909.419] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 1000)
>
> # Run progress: 11.11% complete, ETA 00:04:45
> # Fork: 1 of 1
> # Warmup Iteration 1: 4329.734 ops/s
> Iteration 1: 4115.153 ops/s
> Iteration 2: 3873.048 ops/s
> Iteration 3: 3479.972 ops/s
> Iteration 4: 3225.995 ops/s
> Iteration 5: 3002.072 ops/s
>
>
> Result "measureMTTA":
> 3539.248 ±(99.9%) 1757.523 ops/s [Average]
> (min, avg, max) = (3002.072, 3539.248, 4115.153), stdev = 456.423
> CI (99.9%): [1781.725, 5296.771] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 10000)
>
> # Run progress: 22.22% complete, ETA 00:04:10
> # Fork: 1 of 1
> # Warmup Iteration 1: 44.147 ops/s
> Iteration 1: 46.116 ops/s
> Iteration 2: 54.255 ops/s
> Iteration 3: 57.967 ops/s
> Iteration 4: 56.174 ops/s
> Iteration 5: 58.992 ops/s
>
>
> Result "measureMTTA":
> 54.701 ±(99.9%) 19.743 ops/s [Average]
> (min, avg, max) = (46.116, 54.701, 58.992), stdev = 5.127
> CI (99.9%): [34.957, 74.444] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 100)
>
> # Run progress: 33.33% complete, ETA 00:03:38
> # Fork: 1 of 1
> # Warmup Iteration 1: 106633.191 ops/s
> Iteration 1: 136861.727 ops/s
> Iteration 2: 120490.231 ops/s
> Iteration 3: 106383.718 ops/s
> Iteration 4: 126247.774 ops/s
> Iteration 5: 131648.970 ops/s
>
>
> Result "measureMTTR":
> 124326.484 ±(99.9%) 45198.021 ops/s [Average]
> (min, avg, max) = (106383.718, 124326.484, 136861.727), stdev = 11737.783
> CI (99.9%): [79128.463, 169524.505] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 1000)
>
> # Run progress: 44.44% complete, ETA 00:03:01
> # Fork: 1 of 1
> # Warmup Iteration 1: 4012.815 ops/s
> Iteration 1: 4396.495 ops/s
> Iteration 2: 4086.905 ops/s
> Iteration 3: 4024.187 ops/s
> Iteration 4: 3809.202 ops/s
> Iteration 5: 3551.905 ops/s
>
>
> Result "measureMTTR":
> 3973.739 ±(99.9%) 1216.296 ops/s [Average]
> (min, avg, max) = (3551.905, 3973.739, 4396.495), stdev = 315.868
> CI (99.9%): [2757.443, 5190.034] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 10000)
>
> # Run progress: 55.56% complete, ETA 00:02:25
> # Fork: 1 of 1
> # Warmup Iteration 1: 50.933 ops/s
> Iteration 1: 57.153 ops/s
> Iteration 2: 58.741 ops/s
> Iteration 3: 56.398 ops/s
> Iteration 4: 55.078 ops/s
> Iteration 5: 58.142 ops/s
>
>
> Result "measureMTTR":
> 57.102 ±(99.9%) 5.567 ops/s [Average]
> (min, avg, max) = (55.078, 57.102, 58.741), stdev = 1.446
> CI (99.9%): [51.535, 62.670] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 100)
>
> # Run progress: 66.67% complete, ETA 00:01:49
> # Fork: 1 of 1
> # Warmup Iteration 1: 107724.308 ops/s
> Iteration 1: 121787.913 ops/s
> Iteration 2: 119594.939 ops/s
> Iteration 3: 128441.840 ops/s
> Iteration 4: 134176.538 ops/s
> Iteration 5: 144275.262 ops/s
>
>
> Result "measureMTTS":
> 129655.298 ±(99.9%) 38446.901 ops/s [Average]
> (min, avg, max) = (119594.939, 129655.298, 144275.262), stdev = 9984.538
> CI (99.9%): [91208.397, 168102.199] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 1000)
>
> # Run progress: 77.78% complete, ETA 00:01:12
> # Fork: 1 of 1
> # Warmup Iteration 1: 3738.823 ops/s
> Iteration 1: 4026.955 ops/s
> Iteration 2: 3070.155 ops/s
> Iteration 3: 2706.577 ops/s
> Iteration 4: 3510.103 ops/s
> Iteration 5: 4063.217 ops/s
>
>
> Result "measureMTTS":
> 3475.401 ±(99.9%) 2283.158 ops/s [Average]
> (min, avg, max) = (2706.577, 3475.401, 4063.217), stdev = 592.929
> CI (99.9%): [1192.244, 5758.559] (assumes normal distribution)
>
>
> # JMH 1.15 (released 98 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US
> -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark:
> org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 10000)
>
> # Run progress: 88.89% complete, ETA 00:00:36
> # Fork: 1 of 1
> # Warmup Iteration 1: 56.489 ops/s
> Iteration 1: 59.288 ops/s
> Iteration 2: 59.160 ops/s
> Iteration 3: 56.142 ops/s
> Iteration 4: 53.336 ops/s
> Iteration 5: 53.147 ops/s
>
>
> Result "measureMTTS":
> 56.215 ±(99.9%) 11.524 ops/s [Average]
> (min, avg, max) = (53.147, 56.215, 59.288), stdev = 2.993
> CI (99.9%): [44.691, 67.738] (assumes normal distribution)
>
>
> # Run complete. Total time: 00:05:27
>
> Benchmark (numTasks) Mode Cnt
> Score Error Units
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 100 thrpt 5
> 156249.415 ± 52660.004 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 1000 thrpt 5
> 3539.248 ± 1757.523 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA 10000 thrpt 5
> 54.701 ± 19.743 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 100 thrpt 5
> 124326.484 ± 45198.021 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 1000 thrpt 5
> 3973.739 ± 1216.296 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR 10000 thrpt 5
> 57.102 ± 5.567 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 100 thrpt 5
> 129655.298 ± 38446.901 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 1000 thrpt 5
> 3475.401 ± 2283.158 ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS 10000 thrpt 5
> 56.215 ± 11.524 ops/s
>
> Benchmark result is saved to dist/reports/jmh/results.txt
>
> ```
>
>
> Thanks,
>
> Reza Motamedi
>
>
