Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/#review79566 --- Master (f48d0e1) is green with this patch. ./build-support/jenkins/build.sh However, it appears that it might lack test coverage. I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On April 9, 2015, 7:15 p.m., Maxim Khutornenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32990/ > --- > > (Updated April 9, 2015, 7:15 p.m.) > > > Review request for Aurora and Bill Farner. > > > Bugs: AURORA-1263 > https://issues.apache.org/jira/browse/AURORA-1263 > > > Repository: aurora > > > Description > --- > > Using event collections with "select" avoids unnecessary row explosion and > provdes subsecond execution time for a large job update that used to take > half a minute. > > Before: > ``` > Benchmark (instances) Mode Cnt > Score Error Units > UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 > 15.595 ± 1.056 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 > 3.143 ± 0.259 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 > 1.539 ± 0.075 ops/s > ``` > > After: > ``` > Benchmark (instances) Mode Cnt > ScoreError Units > UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 > 278.405 ± 57.311 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 > 65.386 ± 6.455 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 > 33.353 ± 1.974 ops/s > ``` > > > Diffs > - > > src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java > PRE-CREATION > > src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml > 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d > > Diff: https://reviews.apache.org/r/32990/diff/ > > > Testing > --- > > ./gradlew -Pq build > Manual testing in vagrant. > > > Thanks, > > Maxim Khutornenko > >
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/#review79562 --- Ship it! Ship It! - Bill Farner On April 9, 2015, 7:15 p.m., Maxim Khutornenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32990/ > --- > > (Updated April 9, 2015, 7:15 p.m.) > > > Review request for Aurora and Bill Farner. > > > Bugs: AURORA-1263 > https://issues.apache.org/jira/browse/AURORA-1263 > > > Repository: aurora > > > Description > --- > > Using event collections with "select" avoids unnecessary row explosion and > provdes subsecond execution time for a large job update that used to take > half a minute. > > Before: > ``` > Benchmark (instances) Mode Cnt > Score Error Units > UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 > 15.595 ± 1.056 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 > 3.143 ± 0.259 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 > 1.539 ± 0.075 ops/s > ``` > > After: > ``` > Benchmark (instances) Mode Cnt > ScoreError Units > UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 > 278.405 ± 57.311 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 > 65.386 ± 6.455 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 > 33.353 ± 1.974 ops/s > ``` > > > Diffs > - > > src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java > PRE-CREATION > > src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml > 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d > > Diff: https://reviews.apache.org/r/32990/diff/ > > > Testing > --- > > ./gradlew -Pq build > Manual testing in vagrant. > > > Thanks, > > Maxim Khutornenko > >
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/ --- (Updated April 9, 2015, 7:15 p.m.) Review request for Aurora and Bill Farner. Changes --- Fixed import ordering. Bugs: AURORA-1263 https://issues.apache.org/jira/browse/AURORA-1263 Repository: aurora Description --- Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute. Before: ``` Benchmark (instances) Mode Cnt Score Error Units UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 15.595 ± 1.056 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 3.143 ± 0.259 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 1.539 ± 0.075 ops/s ``` After: ``` Benchmark (instances) Mode CntScore Error Units UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 278.405 ± 57.311 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 65.386 ± 6.455 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 33.353 ± 1.974 ops/s ``` Diffs (updated) - src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d Diff: https://reviews.apache.org/r/32990/diff/ Testing --- ./gradlew -Pq build Manual testing in vagrant. Thanks, Maxim Khutornenko
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/#review79556 --- Master (7e3e7c9) is red with this patch. ./build-support/jenkins/build.sh make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift/thrift-0.9.1' make: Leaving directory `/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift' :api:classesThriftNote: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. :api:checkPython :api:generateThriftEntitiesJava :api:classesThriftEntities :api:compileJava UP-TO-DATE :api:generateThriftResources :api:processResources UP-TO-DATE :api:classes :api:jar :compileJavaNote: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/com/twitter/common/args/apt/cmdline.arg.info.txt.2 :processResources :classes :jar :startScripts :distTar :distZip :assemble :compileJmhJava :processJmhResources UP-TO-DATE :jmhClasses :checkstyleJmh[ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java:67: Wrong order for 'com.twitter.common.inject.Bindings' import. Order should be: java, javax, scala, com, net, org. Each group should be separated by a single blank line. FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':checkstyleJmh'. > Checkstyle rule violations were found. See the report at: > file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/jmh.xml * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. BUILD FAILED Total time: 1 mins 39.707 secs I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On April 9, 2015, 6:43 p.m., Maxim Khutornenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32990/ > --- > > (Updated April 9, 2015, 6:43 p.m.) > > > Review request for Aurora and Bill Farner. > > > Bugs: AURORA-1263 > https://issues.apache.org/jira/browse/AURORA-1263 > > > Repository: aurora > > > Description > --- > > Using event collections with "select" avoids unnecessary row explosion and > provdes subsecond execution time for a large job update that used to take > half a minute. > > Before: > ``` > Benchmark (instances) Mode Cnt > Score Error Units > UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 > 15.595 ± 1.056 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 > 3.143 ± 0.259 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 > 1.539 ± 0.075 ops/s > ``` > > After: > ``` > Benchmark (instances) Mode Cnt > ScoreError Units > UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 > 278.405 ± 57.311 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 > 65.386 ± 6.455 ops/s > UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 > 33.353 ± 1.974 ops/s > ``` > > > Diffs > - > > src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java > PRE-CREATION > > src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml > 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d > > Diff: https://reviews.apache.org/r/32990/diff/ > > > Testing > --- > > ./gradlew -Pq build > Manual testing in vagrant. > > > Thanks, > > Maxim Khutornenko > >
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
> On April 9, 2015, 2:29 a.m., Bill Farner wrote: > > Actually - can you add a benchmark to ThriftApiBenchmarks and include > > results in the description? Should be pretty easy to prove the improvement > > of this. Done. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/#review79467 --- On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32990/ > --- > > (Updated April 8, 2015, 9:45 p.m.) > > > Review request for Aurora and Bill Farner. > > > Bugs: AURORA-1263 > https://issues.apache.org/jira/browse/AURORA-1263 > > > Repository: aurora > > > Description > --- > > Using event collections with "select" avoids unnecessary row explosion and > provdes subsecond execution time for a large job update that used to take > half a minute. > > > Diffs > - > > > src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml > 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d > > Diff: https://reviews.apache.org/r/32990/diff/ > > > Testing > --- > > ./gradlew -Pq build > Manual testing in vagrant. > > > Thanks, > > Maxim Khutornenko > >
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/ --- (Updated April 9, 2015, 6:43 p.m.) Review request for Aurora and Bill Farner. Changes --- Adding jmh benchmark. Bugs: AURORA-1263 https://issues.apache.org/jira/browse/AURORA-1263 Repository: aurora Description (updated) --- Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute. Before: ``` Benchmark (instances) Mode Cnt Score Error Units UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 15.595 ± 1.056 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 3.143 ± 0.259 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 1.539 ± 0.075 ops/s ``` After: ``` Benchmark (instances) Mode CntScore Error Units UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 278.405 ± 57.311 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 65.386 ± 6.455 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 33.353 ± 1.974 ops/s ``` Diffs (updated) - src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d Diff: https://reviews.apache.org/r/32990/diff/ Testing --- ./gradlew -Pq build Manual testing in vagrant. Thanks, Maxim Khutornenko
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/#review79467 --- Actually - can you add a benchmark to ThriftApiBenchmarks and include results in the description? Should be pretty easy to prove the improvement of this. - Bill Farner On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32990/ > --- > > (Updated April 8, 2015, 9:45 p.m.) > > > Review request for Aurora and Bill Farner. > > > Bugs: AURORA-1263 > https://issues.apache.org/jira/browse/AURORA-1263 > > > Repository: aurora > > > Description > --- > > Using event collections with "select" avoids unnecessary row explosion and > provdes subsecond execution time for a large job update that used to take > half a minute. > > > Diffs > - > > > src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml > 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d > > Diff: https://reviews.apache.org/r/32990/diff/ > > > Testing > --- > > ./gradlew -Pq build > Manual testing in vagrant. > > > Thanks, > > Maxim Khutornenko > >
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/#review79466 --- Ship it! Great work! - Bill Farner On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32990/ > --- > > (Updated April 8, 2015, 9:45 p.m.) > > > Review request for Aurora and Bill Farner. > > > Bugs: AURORA-1263 > https://issues.apache.org/jira/browse/AURORA-1263 > > > Repository: aurora > > > Description > --- > > Using event collections with "select" avoids unnecessary row explosion and > provdes subsecond execution time for a large job update that used to take > half a minute. > > > Diffs > - > > > src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml > 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d > > Diff: https://reviews.apache.org/r/32990/diff/ > > > Testing > --- > > ./gradlew -Pq build > Manual testing in vagrant. > > > Thanks, > > Maxim Khutornenko > >
Re: Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/#review79441 --- Master (35f3e8d) is green with this patch. ./build-support/jenkins/build.sh However, it appears that it might lack test coverage. I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32990/ > --- > > (Updated April 8, 2015, 9:45 p.m.) > > > Review request for Aurora and Bill Farner. > > > Bugs: AURORA-1263 > https://issues.apache.org/jira/browse/AURORA-1263 > > > Repository: aurora > > > Description > --- > > Using event collections with "select" avoids unnecessary row explosion and > provdes subsecond execution time for a large job update that used to take > half a minute. > > > Diffs > - > > > src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml > 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d > > Diff: https://reviews.apache.org/r/32990/diff/ > > > Testing > --- > > ./gradlew -Pq build > Manual testing in vagrant. > > > Thanks, > > Maxim Khutornenko > >
Review Request 32990: Improving JobUpdateDetails fetch performance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32990/ --- Review request for Aurora and Bill Farner. Bugs: AURORA-1263 https://issues.apache.org/jira/browse/AURORA-1263 Repository: aurora Description --- Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute. Diffs - src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d Diff: https://reviews.apache.org/r/32990/diff/ Testing --- ./gradlew -Pq build Manual testing in vagrant. Thanks, Maxim Khutornenko
