> On March 7, 2016, 10:36 p.m., Bill Farner wrote: > > api/src/main/thrift/org/apache/aurora/gen/storage.thrift, line 152 > > <https://reviews.apache.org/r/44471/diff/1/?file=1283482#file1283482line152> > > > > What's the motivation behind using `list<string>` here as opposed to > > `string`?
The reason is performance. We can't run the entire script as a single DML statement as it OOMs H2 with any reasonable DB size (e.g. couple thousand of updates). I found there are absolutely no perf issues when the entire script is split into chunks of a few rows per batch. Storing list as opposed to string makes batching easier and safer. > On March 7, 2016, 10:36 p.m., Bill Farner wrote: > > api/src/main/thrift/org/apache/aurora/gen/storage.thrift, line 154 > > <https://reviews.apache.org/r/44471/diff/1/?file=1283482#file1283482line154> > > > > I'm confused by this flag, is it needed or is it the same as `dbScript` > > being set/non-empty? Unfortunately, having a non-empty dbScript is not enough to determine _how_ to handle task store restore. We need to know what's inside of that `dbScript` to properly handle db- vs. mem- task store cases. Consider this example: DBTaskStore: OFF, scheduler running with this patch. The `dbScript` will be populated but it will not have any tasks from TaskStore in it. So, deferring to dbsnapshotting alone would be dangerously wrong here. I have covered possible cases in SnapshotStoreImplIT. Let me know if it's not clear. > On March 7, 2016, 10:36 p.m., Bill Farner wrote: > > src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java, > > line 110 > > <https://reviews.apache.org/r/44471/diff/1/?file=1283491#file1283491line110> > > > > I'd like to avoid a trend of suppressing checkstyle warnings. Consider > > either refactoring to address the warning, or tweak/remove the checkstyle > > rule. I was split on whether to turn that rule off or suppress and decided to suppress instead. This rule does not seem to support Java 8 try-with-resources case well. Extracting a method here would only hurt readability in my opinion. I am fine turning it off as readability issues with nested TRYs are generally caught during code reviews. > On March 7, 2016, 10:36 p.m., Bill Farner wrote: > > src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java, > > line 113 > > <https://reviews.apache.org/r/44471/diff/1/?file=1283491#file1283491line113> > > > > I think i know the answer here, but please add a comment indicating why > > you are not letting mybatis run this query for you. Added comment. - Maxim ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44471/#review122391 ----------------------------------------------------------- On March 7, 2016, 10:22 p.m., Maxim Khutornenko wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44471/ > ----------------------------------------------------------- > > (Updated March 7, 2016, 10:22 p.m.) > > > Review request for Aurora, Bill Farner and Zameer Manji. > > > Bugs: AURORA-1627 > https://issues.apache.org/jira/browse/AURORA-1627 > > > Repository: aurora > > > Description > ------- > > H2 DB is now `script`ed into a dedicated `dbScript` field in `Snapshot` > thrift struct to speed up recovery under H2 MVStore. > > There was extra care needed to ensure task stores are functioning properly > when they are toggled between in-memory and DB mode. Tried to document as > much as possible. Let me know if something is still unclear. The existing > thrift snapshot fields will be supported until all stores are fully migrated > to H2 and removed subsequently along with in-memory store code. > > `SnapshotStoreImplTest` got rewritten into `SnapshotStoreImplIT` to fully > test `TaskStore` migration matrix. > > Also, moved checkstyle suppressions into annotations for more precise control. > > > Diffs > ----- > > api/src/main/thrift/org/apache/aurora/gen/storage.thrift > 6dc46147bb0703e83a210a81ee24081183389a89 > config/checkstyle/checkstyle.xml 2074beb66ca19eb2da3d444bc64e583409e30ead > config/checkstyle/suppressions.xml 50a53e0d96cb365b2182b78769c1749b7e3558a9 > src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java > 60b66e85f83edd3c18c97e3938816cb875249d4e > src/main/java/org/apache/aurora/scheduler/storage/Storage.java > 5124d17c9bc755e33a3e97b914caed34b77cdb94 > > src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java > 46b3d104931f175314902ccd39b81c4b6d67d4f3 > src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java > 6d8fa11d0aed841388e1e87fc0acdaf6b8f42d06 > src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java > cca92dd6e9e52f9e38394c4c688ab4897eb8e23f > src/main/java/org/apache/aurora/scheduler/storage/log/LogStorageModule.java > ed63a7471d654dcefd2ff24e2e462974883541f2 > > src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java > db90150b48c5b134dde6c69f70ebca82bfdd0c12 > > src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java > 2f07afb73f6e585a6b43be68134e2beecac83d31 > src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java > 172dd206d09f131807cb33fe841ca6ebc8198a14 > > src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java > PRE-CREATION > > src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplTest.java > 44078675286dec908da7a404ae8fce7c495a7019 > src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > 75130a315573dbb9eff1a9b1c5408df27623cf54 > > Diff: https://reviews.apache.org/r/44471/diff/ > > > Testing > ------- > > ./gradlew -Pq build > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > > > Thanks, > > Maxim Khutornenko > >
