----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58036/#review170454 -----------------------------------------------------------
Master (c32f14c) is red with this patch. ./build-support/jenkins/build.sh :processResources :classes :jar :startScripts :distTar :distZip :assemble :compileJmhJavaNote: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. :processJmhResources UP-TO-DATE :jmhClasses :checkstyleJmh :jsHint :checkstyleMain[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java:49:8: Unused import - org.apache.aurora.scheduler.storage.db.EnumBackfill. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:29:8: Unused import - org.apache.aurora.gen.CronCollisionPolicy. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:30:8: Unused import - org.apache.aurora.gen.JobUpdateAction. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:31:8: Unused import - org.apache.aurora.gen.JobUpdateStatus. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:32:8: Unused import - org.apache.aurora.gen.MaintenanceMode. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:33:8: Unused import - org.apache.aurora.gen.Mode. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:34:8: Unused import - org.apache.aurora.gen.ScheduleStatus. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java:38:8: Unused import - org.apache.aurora.scheduler.resources.ResourceType. [UnusedImports] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:1: Line does not match expected header line of '^\/\*\*$'. [RegexpHeader] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:17: First sentence should end with a period. [JavadocStyle] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:46: Line is longer than 100 characters (found 102). [LineLength] [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java:54: Line is longer than 100 characters (found 101). [LineLength] FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':checkstyleMain'. > Checkstyle rule violations were found. See the report at: > file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. BUILD FAILED Total time: 1 mins 24.435 secs I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On March 29, 2017, 7:21 p.m., Zameer Manji wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58036/ > ----------------------------------------------------------- > > (Updated March 29, 2017, 7:21 p.m.) > > > Review request for Aurora, Santhosh Kumar Shanmugham and Stephan Erb. > > > Bugs: AURORA-1912 > https://issues.apache.org/jira/browse/AURORA-1912 > > > Repository: aurora > > > Description > ------- > > In our in memory database, we model enums as two column tables. The two > columns > would be `id` which corresponds to the integer value in the thrift enum and > `name` which is the all caps string name of the enum. For example to model the > `JobUpdateStatus` enum we have a table called `job_update_statuses`. In there > the `ROLLING_FORWARD` enum is modeled as a row `(0, "ROLLING_FORWARD")`. Other > tables reference the enum table via the id. > > When we prepare storage on startup the `DbStorage` starts up. It does two > things: > 1. Load in the schema. > 2. Populate the enum tables. > > This ensures that when we insert values into the database, the enum refernces > will be valid. > > However, after we restore from a Snapshot with the `dbScript` field, we blow > all > of that data away and restore what was in the snapshot: > ```` > try (Connection c = ((DataSource) > store.getUnsafeStoreAccess()).getConnection()) { > LOG.info("Dropping all tables"); > try (PreparedStatement drop = c.prepareStatement("DROP ALL OBJECTS")) \ > drop.executeUpdate(); > } > ```` > > This means that if we add a new enum value, and then restore from a snapshot, > that enum value will not exist in the table any more. We could address this by > saying that every enum value addition requires a migration. However instead I > propose not blowing away the work done by `DbStorage` instead and re-hydrating > the enum tables. > > To do this I extracted the logic into a new class `EnumBackfill`. Restoring > from > a snapshot calls this after the migrations are done. The underlying SQL was > changed from `INSERT` to `MERGE` to make this work. > > > Diffs > ----- > > src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java > f0b148cd158d61cd89cc51dca9f3fa4c6feb1b49 > > src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java > 36a1bd5c784ed0febebccfd22e5064f0b2e3106f > src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java > d1a196419b67108ee2bb778f83a2993e2e5ee83b > src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java > 923e904f396724b9dde4a330ef312a6aae2c02a6 > src/main/java/org/apache/aurora/scheduler/storage/db/EnumBackfill.java > PRE-CREATION > > src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java > 81a8cca6974e33c774473a4990e0e981cf6ddee6 > > src/main/resources/org/apache/aurora/scheduler/storage/db/EnumValueMapper.xml > 153fd26c27275c46b190e71d8a5736153f2c2d18 > src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java > 42615da54096d5b06c7989cb30fc3cfbe59bc1b9 > src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java > f26529c76214c8f22563f04a197798c82d341b49 > > src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java > ca9525665805a33b4a322a72022ff037f0dd2a94 > > > Diff: https://reviews.apache.org/r/58036/diff/1/ > > > Testing > ------- > > existing tests and e2e tests > > I also added a new enum value to `JobUpdateStatus` and observed it was > correctly loaded in. > > > Thanks, > > Zameer Manji > >
