> On Jan. 30, 2015, 4:16 p.m., Zameer Manji wrote: > > Ship It!
My comment got chompted, I'm expecting these tests to be run on Jenkins for every commit. - Zameer ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28731/#review70458 ----------------------------------------------------------- On Jan. 29, 2015, 12:11 p.m., Maxim Khutornenko wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/28731/ > ----------------------------------------------------------- > > (Updated Jan. 29, 2015, 12:11 p.m.) > > > Review request for Aurora, Bill Farner and Zameer Manji. > > > Repository: aurora > > > Description > ------- > > Added baseline benchmarks for a few static veto cases. > > > Diffs > ----- > > build.gradle f9f71a84493b782e9f6072e44e89a2c017cf2a09 > src/jmh/java/org/apache/aurora/benchmark/Hosts.java PRE-CREATION > src/jmh/java/org/apache/aurora/benchmark/Offers.java PRE-CREATION > src/jmh/java/org/apache/aurora/benchmark/SchedulerBenchmark.java > 5cecada93e4e04b689e826af49f691ed7e94ae49 > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java > PRE-CREATION > src/jmh/java/org/apache/aurora/benchmark/Tasks.java PRE-CREATION > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java PRE-CREATION > > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java > PRE-CREATION > src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java > PRE-CREATION > src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java > b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 > > src/main/java/org/apache/aurora/scheduler/async/preemptor/CachedClusterState.java > 03c2a8fde4a3fe5ac73f44da2cbe227995501c46 > src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java > f7e157c890b5627411acd4bd5c2559ef4829147c > > src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java > 0204d14b19ae412236f19ca274d81decb4eba12d > > Diff: https://reviews.apache.org/r/28731/diff/ > > > Testing > ------- > > Sample run on a local box: > ``` > # VM invoker: > /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java > # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en > -Duser.variant > # Warmup: 10 iterations, 1 s each > # Measurement: 100 iterations, 1 s each > # Timeout: 10 min per iteration > # Threads: 1 thread, will synchronize iterations > # Benchmark mode: Average time, time/op > # Benchmark: > org.apache.aurora.benchmark.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark > > # Run progress: 0.00% complete, ETA 00:05:30 > # Fork: 1 of 1 > # Warmup Iteration 1: 278284250.000 ns/op > # Warmup Iteration 2: 70294312.500 ns/op > # Warmup Iteration 3: 19293379.310 ns/op > # Warmup Iteration 4: 11945387.097 ns/op > # Warmup Iteration 5: 10725388.350 ns/op > # Warmup Iteration 6: 13043282.353 ns/op > # Warmup Iteration 7: 9233083.333 ns/op > # Warmup Iteration 8: 9521051.724 ns/op > # Warmup Iteration 9: 10750854.369 ns/op > # Warmup Iteration 10: 7460243.243 ns/op > Iteration 1: 7885364.286 ns/op > Iteration 2: 7735139.860 ns/op > Iteration 3: 7660208.333 ns/op > Iteration 4: 7761204.225 ns/op > Iteration 5: 7868907.143 ns/op > Iteration 6: 7567404.110 ns/op > Iteration 7: 7611000.000 ns/op > Iteration 8: 7766154.930 ns/op > Iteration 9: 7669344.828 ns/op > Iteration 10: 7707783.217 ns/op > Iteration 11: 7435651.007 ns/op > Iteration 12: 7697631.944 ns/op > Iteration 13: 7712531.469 ns/op > Iteration 14: 7899407.143 ns/op > Iteration 15: 7448472.973 ns/op > Iteration 16: 7791521.127 ns/op > Iteration 17: 7612213.793 ns/op > Iteration 18: 7710867.133 ns/op > Iteration 19: 7649296.552 ns/op > Iteration 20: 7768309.859 ns/op > Iteration 21: 7688666.667 ns/op > Iteration 22: 7531557.823 ns/op > Iteration 23: 7381193.333 ns/op > Iteration 24: 7726006.993 ns/op > Iteration 25: 7603358.621 ns/op > Iteration 26: 7653631.944 ns/op > Iteration 27: 7442275.168 ns/op > Iteration 28: 7613186.207 ns/op > Iteration 29: 7765823.944 ns/op > Iteration 30: 7489687.075 ns/op > Iteration 31: 7811443.662 ns/op > Iteration 32: 8015007.246 ns/op > Iteration 33: 8192392.593 ns/op > Iteration 34: 8040335.766 ns/op > Iteration 35: 7584212.329 ns/op > Iteration 36: 8001934.783 ns/op > Iteration 37: 9744815.789 ns/op > Iteration 38: 11688284.211 ns/op > Iteration 39: 8661406.250 ns/op > Iteration 40: 7678413.793 ns/op > Iteration 41: 8502223.077 ns/op > Iteration 42: 7640820.690 ns/op > Iteration 43: 7875624.113 ns/op > Iteration 44: 7506809.524 ns/op > Iteration 45: 8005431.655 ns/op > Iteration 46: 8081664.234 ns/op > Iteration 47: 7579438.356 ns/op > Iteration 48: 7993405.797 ns/op > Iteration 49: 7571958.904 ns/op > Iteration 50: 8116463.235 ns/op > Iteration 51: 7941330.935 ns/op > Iteration 52: 7687145.833 ns/op > Iteration 53: 8082554.745 ns/op > Iteration 54: 7597889.655 ns/op > Iteration 55: 7299907.285 ns/op > Iteration 56: 7992789.855 ns/op > Iteration 57: 7648268.966 ns/op > Iteration 58: 7570863.014 ns/op > Iteration 59: 7885078.571 ns/op > Iteration 60: 7647158.621 ns/op > Iteration 61: 7830858.156 ns/op > Iteration 62: 7773690.141 ns/op > Iteration 63: 7905850.000 ns/op > Iteration 64: 7653800.000 ns/op > Iteration 65: 7408248.322 ns/op > Iteration 66: 7961352.518 ns/op > Iteration 67: 7879785.714 ns/op > Iteration 68: 8308616.541 ns/op > Iteration 69: 7396852.349 ns/op > Iteration 70: 8203214.815 ns/op > Iteration 71: 7548821.918 ns/op > Iteration 72: 7834631.206 ns/op > Iteration 73: 7451594.595 ns/op > Iteration 74: 7633262.069 ns/op > Iteration 75: 8182348.148 ns/op > Iteration 76: 7412646.667 ns/op > Iteration 77: 7662375.000 ns/op > Iteration 78: 7714825.175 ns/op > Iteration 79: 7726195.804 ns/op > Iteration 80: 7552568.493 ns/op > Iteration 81: 7642655.172 ns/op > Iteration 82: 7632737.931 ns/op > Iteration 83: 7567616.438 ns/op > Iteration 84: 7641517.241 ns/op > Iteration 85: 7664152.778 ns/op > Iteration 86: 8082532.847 ns/op > Iteration 87: 8027427.536 ns/op > Iteration 88: 7837191.489 ns/op > Iteration 89: 7587938.356 ns/op > Iteration 90: 7631262.069 ns/op > Iteration 91: 7672423.611 ns/op > Iteration 92: 7505258.503 ns/op > Iteration 93: 7410449.664 ns/op > Iteration 94: 8164419.118 ns/op > Iteration 95: 7712930.070 ns/op > Iteration 96: 7934287.770 ns/op > Iteration 97: 8007239.130 ns/op > Iteration 98: 8250343.284 ns/op > Iteration 99: 8482853.846 ns/op > Iteration 100: 8138595.588 ns/op > > > Result: 7831342.453 ±(99.9%) 172179.897 ns/op [Average] > Statistics: (min, avg, max) = (7299907.285, 7831342.453, 11688284.211), > stdev = 507676.348 > Confidence interval (99.9%): [7659162.555, 8003522.350] > > > # VM invoker: > /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java > # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en > -Duser.variant > # Warmup: 10 iterations, 1 s each > # Measurement: 100 iterations, 1 s each > # Timeout: 10 min per iteration > # Threads: 1 thread, will synchronize iterations > # Benchmark mode: Average time, time/op > # Benchmark: > org.apache.aurora.benchmark.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark > > # Run progress: 33.33% complete, ETA 00:04:43 > # Fork: 1 of 1 > # Warmup Iteration 1: 130891625.000 ns/op > # Warmup Iteration 2: 18382600.000 ns/op > # Warmup Iteration 3: 8235328.358 ns/op > # Warmup Iteration 4: 8724039.370 ns/op > # Warmup Iteration 5: 5823163.158 ns/op > # Warmup Iteration 6: 5425686.275 ns/op > # Warmup Iteration 7: 5722766.839 ns/op > # Warmup Iteration 8: 5716419.689 ns/op > # Warmup Iteration 9: 6278869.318 ns/op > # Warmup Iteration 10: 5738276.042 ns/op > Iteration 1: 5761427.083 ns/op > Iteration 2: 5946838.710 ns/op > Iteration 3: 6091066.298 ns/op > Iteration 4: 5721787.565 ns/op > Iteration 5: 6785852.761 ns/op > Iteration 6: 5813373.684 ns/op > Iteration 7: 5866824.468 ns/op > Iteration 8: 6948911.950 ns/op > Iteration 9: 6042060.109 ns/op > Iteration 10: 5633122.449 ns/op > Iteration 11: 5753838.542 ns/op > Iteration 12: 5853433.862 ns/op > Iteration 13: 5752322.917 ns/op > Iteration 14: 5332053.140 ns/op > Iteration 15: 5318278.846 ns/op > Iteration 16: 5260276.190 ns/op > Iteration 17: 5700340.206 ns/op > Iteration 18: 5767635.417 ns/op > Iteration 19: 5367514.563 ns/op > Iteration 20: 5721994.819 ns/op > Iteration 21: 5711243.523 ns/op > Iteration 22: 6276647.727 ns/op > Iteration 23: 5934473.118 ns/op > Iteration 24: 5821626.316 ns/op > Iteration 25: 5533788.945 ns/op > Iteration 26: 7226183.007 ns/op > Iteration 27: 5448920.792 ns/op > Iteration 28: 5749453.125 ns/op > Iteration 29: 5906727.273 ns/op > Iteration 30: 5702489.691 ns/op > Iteration 31: 5355660.194 ns/op > Iteration 32: 7413523.490 ns/op > Iteration 33: 7467831.081 ns/op > Iteration 34: 8952487.805 ns/op > Iteration 35: 8635742.188 ns/op > Iteration 36: 7981898.551 ns/op > Iteration 37: 7761732.394 ns/op > Iteration 38: 7130438.710 ns/op > Iteration 39: 8349406.015 ns/op > Iteration 40: 7980985.612 ns/op > Iteration 41: 8037847.826 ns/op > Iteration 42: 6315828.571 ns/op > Iteration 43: 5375892.683 ns/op > Iteration 44: 5897005.348 ns/op > Iteration 45: 5568793.970 ns/op > Iteration 46: 5273685.714 ns/op > Iteration 47: 8879392.000 ns/op > Iteration 48: 8641343.750 ns/op > Iteration 49: 10007136.364 ns/op > Iteration 50: 8484908.397 ns/op > Iteration 51: 7659333.333 ns/op > Iteration 52: 8661968.750 ns/op > Iteration 53: 8923153.226 ns/op > Iteration 54: 8746527.559 ns/op > Iteration 55: 8981138.211 ns/op > Iteration 56: 5631193.878 ns/op > Iteration 57: 5945069.892 ns/op > Iteration 58: 5610161.616 ns/op > Iteration 59: 5348665.049 ns/op > Iteration 60: 5429064.039 ns/op > Iteration 61: 5432418.719 ns/op > Iteration 62: 5621515.306 ns/op > Iteration 63: 5268630.332 ns/op > Iteration 64: 5849894.180 ns/op > Iteration 65: 5418126.829 ns/op > Iteration 66: 6133177.778 ns/op > Iteration 67: 6195095.506 ns/op > Iteration 68: 6111276.243 ns/op > Iteration 69: 5886244.681 ns/op > Iteration 70: 6430941.860 ns/op > Iteration 71: 5783329.843 ns/op > Iteration 72: 6507141.176 ns/op > Iteration 73: 10092783.784 ns/op > Iteration 74: 11873978.495 ns/op > Iteration 75: 8876424.000 ns/op > Iteration 76: 5878223.404 ns/op > Iteration 77: 7130651.613 ns/op > Iteration 78: 5508910.000 ns/op > Iteration 79: 5481338.308 ns/op > Iteration 80: 5999592.391 ns/op > Iteration 81: 6065917.582 ns/op > Iteration 82: 6087812.155 ns/op > Iteration 83: 5653512.821 ns/op > Iteration 84: 5528065.000 ns/op > Iteration 85: 5599314.721 ns/op > Iteration 86: 5300822.115 ns/op > Iteration 87: 6341833.333 ns/op > Iteration 88: 5511190.000 ns/op > Iteration 89: 5813026.316 ns/op > Iteration 90: 5914112.299 ns/op > Iteration 91: 5663928.205 ns/op > Iteration 92: 5723891.192 ns/op > Iteration 93: 5580843.434 ns/op > Iteration 94: 5990407.609 ns/op > Iteration 95: 5656717.949 ns/op > Iteration 96: 5432532.020 ns/op > Iteration 97: 5797827.225 ns/op > Iteration 98: 5647410.256 ns/op > Iteration 99: 5250814.286 ns/op > Iteration 100: 5602177.665 ns/op > > > Result: 6418401.739 ±(99.9%) 447052.433 ns/op [Average] > Statistics: (min, avg, max) = (5250814.286, 6418401.739, 11873978.495), > stdev = 1318144.279 > Confidence interval (99.9%): [5971349.307, 6865454.172] > > > # VM invoker: > /Library/Java/JavaVirtualMachines/jdk1.7.0_25.jdk/Contents/Home/jre/bin/java > # VM options: -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en > -Duser.variant > # Warmup: 10 iterations, 1 s each > # Measurement: 100 iterations, 1 s each > # Timeout: 10 min per iteration > # Threads: 1 thread, will synchronize iterations > # Benchmark mode: Average time, time/op > # Benchmark: > org.apache.aurora.benchmark.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark > > # Run progress: 66.67% complete, ETA 00:02:19 > # Fork: 1 of 1 > # Warmup Iteration 1: 7780760000.000 ns/op > # Warmup Iteration 2: 165927428.571 ns/op > # Warmup Iteration 3: 94242076.923 ns/op > # Warmup Iteration 4: 66061000.000 ns/op > # Warmup Iteration 5: 70444500.000 ns/op > # Warmup Iteration 6: 62350000.000 ns/op > # Warmup Iteration 7: 63746555.556 ns/op > # Warmup Iteration 8: 62712111.111 ns/op > # Warmup Iteration 9: 62776388.889 ns/op > # Warmup Iteration 10: 62455277.778 ns/op > Iteration 1: 65696764.706 ns/op > Iteration 2: 66255823.529 ns/op > Iteration 3: 68231647.059 ns/op > Iteration 4: 71606750.000 ns/op > Iteration 5: 66780235.294 ns/op > Iteration 6: 69914750.000 ns/op > Iteration 7: 74681333.333 ns/op > Iteration 8: 64133944.444 ns/op > Iteration 9: 60831736.842 ns/op > Iteration 10: 63261444.444 ns/op > Iteration 11: 60814105.263 ns/op > Iteration 12: 67882941.176 ns/op > Iteration 13: 62512888.889 ns/op > Iteration 14: 65551823.529 ns/op > Iteration 15: 65574294.118 ns/op > Iteration 16: 67928352.941 ns/op > Iteration 17: 65510588.235 ns/op > Iteration 18: 62637055.556 ns/op > Iteration 19: 62900222.222 ns/op > Iteration 20: 63715944.444 ns/op > Iteration 21: 62284611.111 ns/op > Iteration 22: 68174235.294 ns/op > Iteration 23: 69029187.500 ns/op > Iteration 24: 68584764.706 ns/op > Iteration 25: 63332777.778 ns/op > Iteration 26: 64316000.000 ns/op > Iteration 27: 64219444.444 ns/op > Iteration 28: 64219888.889 ns/op > Iteration 29: 63113888.889 ns/op > Iteration 30: 65802117.647 ns/op > Iteration 31: 63661277.778 ns/op > Iteration 32: 63322000.000 ns/op > Iteration 33: 63770000.000 ns/op > Iteration 34: 63920333.333 ns/op > Iteration 35: 62403500.000 ns/op > Iteration 36: 64137166.667 ns/op > Iteration 37: 64725444.444 ns/op > Iteration 38: 63145166.667 ns/op > Iteration 39: 64183944.444 ns/op > Iteration 40: 65873666.667 ns/op > Iteration 41: 65648529.412 ns/op > Iteration 42: 66487176.471 ns/op > Iteration 43: 61702888.889 ns/op > Iteration 44: 63267666.667 ns/op > Iteration 45: 69614125.000 ns/op > Iteration 46: 61574222.222 ns/op > Iteration 47: 62678611.111 ns/op > Iteration 48: 62792277.778 ns/op > Iteration 49: 65355352.941 ns/op > Iteration 50: 65206823.529 ns/op > Iteration 51: 64246722.222 ns/op > Iteration 52: 68074823.529 ns/op > Iteration 53: 67314823.529 ns/op > Iteration 54: 65511882.353 ns/op > Iteration 55: 63607722.222 ns/op > Iteration 56: 66646529.412 ns/op > Iteration 57: 67675764.706 ns/op > Iteration 58: 62502388.889 ns/op > Iteration 59: 70476625.000 ns/op > Iteration 60: 97138500.000 ns/op > Iteration 61: 93276461.538 ns/op > Iteration 62: 103508818.182 ns/op > Iteration 63: 96725916.667 ns/op > Iteration 64: 97539500.000 ns/op > Iteration 65: 93591461.538 ns/op > Iteration 66: 95847083.333 ns/op > Iteration 67: 101147666.667 ns/op > Iteration 68: 91144230.769 ns/op > Iteration 69: 69291375.000 ns/op > Iteration 70: 67021941.176 ns/op > Iteration 71: 64101166.667 ns/op > Iteration 72: 66573823.529 ns/op > Iteration 73: 66116235.294 ns/op > Iteration 74: 62875611.111 ns/op > Iteration 75: 72765812.500 ns/op > Iteration 76: 65663176.471 ns/op > Iteration 77: 64187000.000 ns/op > Iteration 78: 64425444.444 ns/op > Iteration 79: 63835111.111 ns/op > Iteration 80: 62942166.667 ns/op > Iteration 81: 69006500.000 ns/op > Iteration 82: 64798176.471 ns/op > Iteration 83: 67823117.647 ns/op > Iteration 84: 77090866.667 ns/op > Iteration 85: 71548625.000 ns/op > Iteration 86: 63718333.333 ns/op > Iteration 87: 67034764.706 ns/op > Iteration 88: 61701833.333 ns/op > Iteration 89: 65456529.412 ns/op > Iteration 90: 63462388.889 ns/op > Iteration 91: 61797555.556 ns/op > Iteration 92: 66772823.529 ns/op > Iteration 93: 66240117.647 ns/op > Iteration 94: 63701444.444 ns/op > Iteration 95: 64047888.889 ns/op > Iteration 96: 62162777.778 ns/op > Iteration 97: 64846588.235 ns/op > Iteration 98: 61865666.667 ns/op > Iteration 99: 61965222.222 ns/op > Iteration 100: 64089333.333 ns/op > > > Result: 68178760.746 ±(99.9%) 3225821.640 ns/op [Average] > Statistics: (min, avg, max) = (60814105.263, 68178760.746, 103508818.182), > stdev = 9511408.567 > Confidence interval (99.9%): [64952939.106, 71404582.386] > > > # Run complete. Total time: 00:07:18 > > Benchmark > Mode Samples Score Error Units > o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark > avgt 100 7831342.453 ± 172179.897 ns/op > o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark > avgt 100 6418401.739 ± 447052.433 ns/op > o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark > avgt 100 68178760.746 ± 3225821.640 ns/op > ``` > > > Thanks, > > Maxim Khutornenko > >
