> On March 7, 2016, 10:36 p.m., Bill Farner wrote:
> > api/src/main/thrift/org/apache/aurora/gen/storage.thrift, line 152
> > <https://reviews.apache.org/r/44471/diff/1/?file=1283482#file1283482line152>
> >
> >     What's the motivation behind using `list<string>` here as opposed to 
> > `string`?

The reason is performance. We can't run the entire script as a single DML 
statement as it OOMs H2 with any reasonable DB size (e.g. couple thousand of 
updates). I found there are absolutely no perf issues when the entire script is 
split into chunks of a few rows per batch. Storing list as opposed to string 
makes batching easier and safer.


> On March 7, 2016, 10:36 p.m., Bill Farner wrote:
> > api/src/main/thrift/org/apache/aurora/gen/storage.thrift, line 154
> > <https://reviews.apache.org/r/44471/diff/1/?file=1283482#file1283482line154>
> >
> >     I'm confused by this flag, is it needed or is it the same as `dbScript` 
> > being set/non-empty?

Unfortunately, having a non-empty dbScript is not enough to determine _how_ to 
handle task store restore. We need to know what's inside of that `dbScript` to 
properly handle db- vs. mem- task store cases. Consider this example: 
DBTaskStore: OFF, scheduler running with this patch. The `dbScript` will be 
populated but it will not have any tasks from TaskStore in it. So, deferring to 
dbsnapshotting alone would be dangerously wrong here.

I have covered possible cases in SnapshotStoreImplIT. Let me know if it's not 
clear.


> On March 7, 2016, 10:36 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java,
> >  line 110
> > <https://reviews.apache.org/r/44471/diff/1/?file=1283491#file1283491line110>
> >
> >     I'd like to avoid a trend of suppressing checkstyle warnings.  Consider 
> > either refactoring to address the warning, or tweak/remove the checkstyle 
> > rule.

I was split on whether to turn that rule off or suppress and decided to 
suppress instead. This rule does not seem to support Java 8 try-with-resources 
case well. Extracting a method here would only hurt readability in my opinion. 
I am fine turning it off as readability issues with nested TRYs are generally 
caught during code reviews.


> On March 7, 2016, 10:36 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java,
> >  line 113
> > <https://reviews.apache.org/r/44471/diff/1/?file=1283491#file1283491line113>
> >
> >     I think i know the answer here, but please add a comment indicating why 
> > you are not letting mybatis run this query for you.

Added comment.


- Maxim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44471/#review122391
-----------------------------------------------------------


On March 7, 2016, 10:22 p.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44471/
> -----------------------------------------------------------
> 
> (Updated March 7, 2016, 10:22 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Bugs: AURORA-1627
>     https://issues.apache.org/jira/browse/AURORA-1627
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> H2 DB is now `script`ed into a dedicated `dbScript` field in `Snapshot` 
> thrift struct to speed up recovery under H2 MVStore.
> 
> There was extra care needed to ensure task stores are functioning properly 
> when they are toggled between in-memory and DB mode. Tried to document as 
> much as possible. Let me know if something is still unclear. The existing 
> thrift snapshot fields will be supported until all stores are fully migrated 
> to H2 and removed subsequently along with in-memory store code.
> 
> `SnapshotStoreImplTest` got rewritten into `SnapshotStoreImplIT` to fully 
> test `TaskStore` migration matrix.
> 
> Also, moved checkstyle suppressions into annotations for more precise control.
> 
> 
> Diffs
> -----
> 
>   api/src/main/thrift/org/apache/aurora/gen/storage.thrift 
> 6dc46147bb0703e83a210a81ee24081183389a89 
>   config/checkstyle/checkstyle.xml 2074beb66ca19eb2da3d444bc64e583409e30ead 
>   config/checkstyle/suppressions.xml 50a53e0d96cb365b2182b78769c1749b7e3558a9 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 60b66e85f83edd3c18c97e3938816cb875249d4e 
>   src/main/java/org/apache/aurora/scheduler/storage/Storage.java 
> 5124d17c9bc755e33a3e97b914caed34b77cdb94 
>   
> src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java
>  46b3d104931f175314902ccd39b81c4b6d67d4f3 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 
> 6d8fa11d0aed841388e1e87fc0acdaf6b8f42d06 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java 
> cca92dd6e9e52f9e38394c4c688ab4897eb8e23f 
>   src/main/java/org/apache/aurora/scheduler/storage/log/LogStorageModule.java 
> ed63a7471d654dcefd2ff24e2e462974883541f2 
>   
> src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java 
> db90150b48c5b134dde6c69f70ebca82bfdd0c12 
>   
> src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java 
> 2f07afb73f6e585a6b43be68134e2beecac83d31 
>   src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java 
> 172dd206d09f131807cb33fe841ca6ebc8198a14 
>   
> src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java
>  PRE-CREATION 
>   
> src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplTest.java
>  44078675286dec908da7a404ae8fce7c495a7019 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> 75130a315573dbb9eff1a9b1c5408df27623cf54 
> 
> Diff: https://reviews.apache.org/r/44471/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew -Pq build
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Reply via email to