----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64519/#review193479 -----------------------------------------------------------
Reviewer notes src/main/java/org/apache/aurora/scheduler/storage/durability/DurableStorageModule.java Lines 27 (patched) <https://reviews.apache.org/r/64519/#comment272036> This refactor was ported from [r/64288/](https://reviews.apache.org/r/64288/) to reduce the necessary surface area in `DataCompatibilityTest`. src/test/java/org/apache/aurora/scheduler/storage/durability/DataCompatibilityTest.java Lines 113 (patched) <https://reviews.apache.org/r/64519/#comment272037> This (hopefully) solves the minimization of manual labor. I was not pleased with the idea of hand-rolling fully-described objects for each type, as they would be surely become outdated. src/test/java/org/apache/aurora/scheduler/storage/durability/DataCompatibilityTest.java Lines 126 (patched) <https://reviews.apache.org/r/64519/#comment272038> Some types have specific requirements for recovery. In this case, recovery will fail if a `SaveQuota` does not have (cpu, ram, disk) resources. To accommodate this, it's possible to mix-and-match manually-described structs and auto-generated ones. src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/pruneJobUpdateHistory Lines 1 (patched) <https://reviews.apache.org/r/64519/#comment272040> Here begins the golden files. These are all pretty-printed `TJSONProtocol`. While it's not easy to look at one of these and know the structure, it is fairly straightforward to map a thrift change to a golden file change. - Bill Farner On Dec. 11, 2017, 5:35 p.m., Bill Farner wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64519/ > ----------------------------------------------------------- > > (Updated Dec. 11, 2017, 5:35 p.m.) > > > Review request for Aurora, David McLaughlin, Jordan Ly, and Stephan Erb. > > > Repository: aurora > > > Description > ------- > > This is intended as a safeguard against future compatibility regressions like > [AURORA-1959](https://issues.apache.org/jira/browse/AURORA-1959). > > I approached this with a few goals: > > - golden files should be text-based and human-readable. This allows for > non-opaque code reviews, and simpler remedy when it's necessary to update the > goldens (i.e. copy-pasteable) > - guidance for schema evolution should be included directly in test failures > - separate detection of 'what the scheduler _can_ read' and 'what the > scheduler writes' > - reasonably-complete schema coverage with minimal manual labor. These > tests auto-generate structs to mitigate maintenance burden of test code as > schemas evolve. > > This is not a replacement for vigilance with data compatibility, but it > should at least > > 1. mitigate unintentional breakages in compatibility, especially for new > contributors > 2. draw code reviewer attention to compatibility changes in a patch (signaled > by changes to golden files) > > > Diffs > ----- > > src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java > 2bf7e7ba414d36c99a49599fdc8cf3abdc945dc9 > src/main/java/org/apache/aurora/scheduler/config/CliOptions.java > b7f43e0d6efbddcac640c3d39c7bc56400e68e68 > > src/main/java/org/apache/aurora/scheduler/storage/durability/DurableStorageModule.java > PRE-CREATION > src/main/java/org/apache/aurora/scheduler/storage/log/LogStorageModule.java > 75ec42aad0b822d6c3dcd5b1307a4fcb86caa5c0 > src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java > 4929ecdec90a1ccbcafa4857dea83cec1e2d7fd4 > > src/test/java/org/apache/aurora/scheduler/storage/durability/DataCompatibilityTest.java > PRE-CREATION > src/test/java/org/apache/aurora/scheduler/storage/durability/Generator.java > PRE-CREATION > > src/test/java/org/apache/aurora/scheduler/storage/log/NonVolatileStorageTest.java > eb966d722dc01d1760566bc57358afac722d5fec > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/pruneJobUpdateHistory > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/removeJob > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/removeJobUpdate > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/removeLock > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/removeQuota > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/removeTasks > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveCronJob > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveFrameworkId > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveHostAttributes > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveJobInstanceUpdateEvent > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveJobUpdate > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveJobUpdateEvent > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveLock > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveQuota > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/current/saveTasks > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/1-pruneJobUpdateHistory > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/10-saveJobUpdate > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/11-saveJobUpdateEvent > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/12-saveJobInstanceUpdateEvent > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/13-saveLock > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/14-saveQuota > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/15-saveTasks > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/2-removeJob > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/3-removeJobUpdate > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/4-removeLock > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/5-removeQuota > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/6-removeTasks > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/7-saveCronJob > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/8-saveFrameworkId > PRE-CREATION > > src/test/resources/org/apache/aurora/scheduler/storage/durability/goldens/read-compatible/9-saveHostAttributes > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64519/diff/1/ > > > Testing > ------- > > > Thanks, > > Bill Farner > >
