----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44471/#review122541 -----------------------------------------------------------
Ship it! I have one objection to the future of this patch which is dropping the other fields in the snapshot after we move all in memory storage to H2. I think keeping those fields for a while will be useful because they allow cluster operators to look at a snapshot of the cluster state and perform analytics, etc. - Zameer Manji On March 7, 2016, 3:55 p.m., Maxim Khutornenko wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44471/ > ----------------------------------------------------------- > > (Updated March 7, 2016, 3:55 p.m.) > > > Review request for Aurora, Bill Farner and Zameer Manji. > > > Bugs: AURORA-1627 > https://issues.apache.org/jira/browse/AURORA-1627 > > > Repository: aurora > > > Description > ------- > > H2 DB is now `script`ed into a dedicated `dbScript` field in `Snapshot` > thrift struct to speed up recovery under H2 MVStore. > > There was extra care needed to ensure task stores are functioning properly > when they are toggled between in-memory and DB mode. Tried to document as > much as possible. Let me know if something is still unclear. The existing > thrift snapshot fields will be supported until all stores are fully migrated > to H2 and removed subsequently along with in-memory store code. > > `SnapshotStoreImplTest` got rewritten into `SnapshotStoreImplIT` to fully > test `TaskStore` migration matrix. > > Also, moved checkstyle suppressions into annotations for more precise control. > > Master: > ``` > Benchmark (updateCount) > Mode Cnt Score Error Units > SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run 1 > thrpt 5 4.876 ± 0.403 ops/s > SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run 5 > thrpt 5 0.942 ± 0.032 ops/s > SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run 10 > thrpt 5 0.436 ± 0.040 ops/s > ``` > > This patch: > ``` > Benchmark (updateCount) > Mode Cnt Score Error Units > SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run 1 > thrpt 5 9.846 ± 0.886 ops/s > SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run 5 > thrpt 5 1.977 ± 0.319 ops/s > SnapshotBenchmarks.RestoreSnapshotWithUpdatesBenchmark.run 10 > thrpt 5 1.310 ± 0.139 ops/s > ``` > > > Diffs > ----- > > api/src/main/thrift/org/apache/aurora/gen/storage.thrift > 6dc46147bb0703e83a210a81ee24081183389a89 > config/checkstyle/checkstyle.xml 2074beb66ca19eb2da3d444bc64e583409e30ead > src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java > 50044e1c38c712a52855dd96fa2a9de769d6c973 > src/jmh/java/org/apache/aurora/benchmark/SnapshotBenchmarks.java > ca484fab2a9b136c6d5b9be31e1ad1a5360f1b7a > src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java > 92849d9a82dcd6d5fe6cc452061bcdca31415f40 > src/main/java/org/apache/aurora/scheduler/storage/Storage.java > 5124d17c9bc755e33a3e97b914caed34b77cdb94 > > src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java > 46b3d104931f175314902ccd39b81c4b6d67d4f3 > src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java > 6d8fa11d0aed841388e1e87fc0acdaf6b8f42d06 > src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java > cca92dd6e9e52f9e38394c4c688ab4897eb8e23f > src/main/java/org/apache/aurora/scheduler/storage/log/LogStorageModule.java > ed63a7471d654dcefd2ff24e2e462974883541f2 > > src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java > db90150b48c5b134dde6c69f70ebca82bfdd0c12 > > src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java > 2f07afb73f6e585a6b43be68134e2beecac83d31 > src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java > 172dd206d09f131807cb33fe841ca6ebc8198a14 > > src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java > PRE-CREATION > > src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplTest.java > 44078675286dec908da7a404ae8fce7c495a7019 > src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > 75130a315573dbb9eff1a9b1c5408df27623cf54 > > Diff: https://reviews.apache.org/r/44471/diff/ > > > Testing > ------- > > ./gradlew -Pq build > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > > > Thanks, > > Maxim Khutornenko > >
