Re: Review Request 34440: Implementing task reconciler.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/#review84990 --- Master (998993d) is red with this patch. ./build-support/jenkins/build.sh :api:checkPython :api:generateThriftEntitiesJava :api:classesThriftEntities :api:compileJava UP-TO-DATE :api:generateThriftResources :api:processResources UP-TO-DATE :api:classes :api:jar :compileJavaNote: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/com/twitter/common/args/apt/cmdline.arg.info.txt.2 :processResources :classes :jar :startScripts :distTar :distZip :assemble :compileJmhJavaNote: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. :processJmhResources UP-TO-DATE :jmhClasses :checkstyleJmh :jsHint :checkstyleMain[ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java:37: 'com.twitter.common.quantity.Time.MINUTES' should be separated from previous imports. FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':checkstyleMain'. Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.xml * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. BUILD FAILED Total time: 1 mins 53.619 secs I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On May 22, 2015, 8:47 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 22, 2015, 8:47 p.m.) Review request for Aurora, Ben Mahler, Joshua Cohen, and Zameer Manji. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 34566: Adding H2 management console.
On May 21, 2015, 11:34 p.m., Kevin Sweeney wrote: Have you investigated using the [Console Servlet](http://www.h2database.com/html/tutorial.html) directly? Then we can use the Shiro filter directly and avoid adding another mechanism to configure security as well as avoid adding another listening port. Maxim Khutornenko wrote: That's what I run locally to connect to H2 TCP server. Embedding H2 console directly into the scheduler would not help as it does not support any other security mechanisms besides basic auth (AFAICT). Theoretically, we could put TCP server behind Shiro but then we would face the same problem of client not supporting kerberos. Kevin Sweeney wrote: I might be missing something here. You're connecting to a management console via a browser, right? The console servlet doesn't need to know anything about the filters in front of it, so if your browser supports whatever auth mechanism the filter requires you're in business. If you've got the servlet setup somewhere with ```java serve(/h2console/*).with(H2Console.class); ``` somewhere you can add something like ```java install(ShiroWebModule.guiceFilter(/h2console/*); addFilterChain(/h2console/**, ShiroWebModule.NO_SESSION_CREATION, ShiroWebModule.AUTHC_BASIC, config(ShiroWebModule.PERMS, h2:console)); ``` to `ApiSecurityModule` and users can login with their existing credentials, and configure ACLs around `h2:console` in the same way as they configure all other ACLs. Why introduce another way to do it? Maxim Khutornenko wrote: The servlet is hosted within the browser but it still connects to the TCP server that we have to expose for it to connect to. So, the way it works now (referring to the boxes in http://www.h2database.com/html/tutorial.html): a. Web Browser: loads a servlet UI, which has Connect button to connect to the TCP server; b. H2 console server: that's the TCP server endpoint (in DBModule) that proxies requests from the servlet to the H2; c. H2: our in-memory DB I guess we could potentially embed (a) into scheduler (subject to verificaiont) and hide it behind Shiro. However, it would still have to connect to a TCP endpoint (b), so we would also need to add Shiro there as well. The problem (as I see it) is how to relay credentials from (a) to (b) as servlet opens its own connection to talk to (b) where kerberos headers will not be available. Why introduce another way to do it? No reason at all. I hate dealing with DB credentials, just don't see a way to make it work without hacking into the servlet itself. Maxim Khutornenko wrote: AFAICT, there are only two ways to connect to the H2 DB: AUTO_SERVER and TCP server. I explored the AUTO_SERVER earlier but unfortunately it does not work for in-memory DBs (http://h2database.com/html/features.html#auto_mixed_mode): Multiple processes can access the same database without having to start the server manually. To do that, append ;AUTO_SERVER=TRUE to the database URL. You can use the same database URL independent of whether the database is already open or not. This feature doesn't work with in-memory databases. So, all we left with is a H2 dedicated server instance. Maxim Khutornenko wrote: Actually, looking at their sources it appears that WebServlet creates its own WebServer instance, which in turn may serve the WebApp. This is promising. Will give it a try and report back. Thanks for the nudge. The servlet works just fine in embedded mode against in-memory DB (contrary to what I read elsewhere). I had some concerns about the servlet creating its own instance of WebServer (as that would easily bypass our security model) but turned out it's being used to serve content within the servlet web context. Even though a new http port is assigned, it's not being backed up by the native socket and not accepting connections. Now to figure out how to wire in Shiro filters... - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34566/#review84823 --- On May 21, 2015, 9:15 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34566/ --- (Updated May 21, 2015, 9:15 p.m.) Review request for Aurora, Joshua Cohen and Kevin Sweeney. Bugs: AURORA-1287 https://issues.apache.org/jira/browse/AURORA-1287 Repository: aurora Description --- Adding support for connecting to H2 DB via management console: http://www.h2database.com/html/quickstart.html
Re: Review Request 34440: Implementing task reconciler.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/#review84992 --- src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java https://reviews.apache.org/r/34440/#comment136454 Add @Positive here as well. - Joshua Cohen On May 22, 2015, 8:47 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 22, 2015, 8:47 p.m.) Review request for Aurora, Ben Mahler, Joshua Cohen, and Zameer Manji. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 34440: Implementing task reconciler.
On May 22, 2015, 8:52 p.m., Joshua Cohen wrote: src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java, line 186 https://reviews.apache.org/r/34440/diff/2-3/?file=964680#file964680line186 Add @Positive here as well. Well, this can and should be 0 as well. We start reconciler service on scheduler active, so no reason to delay it further once the feature is active. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/#review84992 --- On May 22, 2015, 8:47 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 22, 2015, 8:47 p.m.) Review request for Aurora, Ben Mahler, Joshua Cohen, and Zameer Manji. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 34440: Implementing task reconciler.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 22, 2015, 8:56 p.m.) Review request for Aurora, Ben Mahler, Joshua Cohen, and Zameer Manji. Changes --- Adding space between imports. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs (updated) - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 34440: Implementing task reconciler.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/#review84999 --- Ship it! Master (998993d) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On May 22, 2015, 8:56 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 22, 2015, 8:56 p.m.) Review request for Aurora, Ben Mahler, Joshua Cohen, and Zameer Manji. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 34440: Implementing task reconciler.
On May 19, 2015, 11:48 p.m., Zameer Manji wrote: Does it make sense for the reconciler to run in parallel with the GC executor mechanism? It seems fine to me, but I would like some re-assurance here. Maxim Khutornenko wrote: GC executor is not adding anything when task reconciliation is ON. Is there a particular use case you have in mind? Zameer Manji wrote: No, I just wanted confirmation that the GC executor causes no harm when reconciliation is ON. No harm if GC executor is ON except for more load on the scheduler. On May 19, 2015, 11:48 p.m., Zameer Manji wrote: src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java, line 43 https://reviews.apache.org/r/34440/diff/1/?file=964438#file964438line43 Please rename TIME to something that is more representitive of what it holds. Maxim Khutornenko wrote: This holds what it tells (Time enum value). Any suggestions? Zameer Manji wrote: TIME_UNIT? Changed to static import. On May 19, 2015, 11:48 p.m., Zameer Manji wrote: src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java, line 186 https://reviews.apache.org/r/34440/diff/1/?file=964437#file964437line186 This seems a little hacky, is there a reason why we cannot have an --enable_reconciliation flag? If disabled we can bind TaskReconciler to an implementation that does nothing. Maxim Khutornenko wrote: I am hesitant adding yet another flag to accomplish what can already be done. This feature is not expected to be optional once GC executor is gone, so I don't see much motivation for extra complexity. Happy to reconsider though if others feel the same. Zameer Manji wrote: Depending on the timeline for the GC executor removal, I see the benefit of a simple binary flag. If 0.9.0 will offer both mechanisms then I think a binary flag is required. If 0.9.0 will only offer reconciliation then I think this is fine. I fully expect 0.9.0 to not have GC executor in place. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/#review84421 --- On May 20, 2015, 1:34 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 20, 2015, 1:34 a.m.) Review request for Aurora, Ben Mahler, Kevin Sweeney, and Zameer Manji. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 34440: Implementing task reconciler.
On May 22, 2015, 8:52 p.m., Joshua Cohen wrote: src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java, line 186 https://reviews.apache.org/r/34440/diff/2-3/?file=964680#file964680line186 Add @Positive here as well. Maxim Khutornenko wrote: Well, this can and should be 0 as well. We start reconciler service on scheduler active, so no reason to delay it further once the feature is active. ack. - Joshua --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/#review84992 --- On May 22, 2015, 8:56 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 22, 2015, 8:56 p.m.) Review request for Aurora, Ben Mahler, Joshua Cohen, and Zameer Manji. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 34501: Defaulting TemporaryStorage to in-memory task store.
On May 20, 2015, 10:17 p.m., Kevin Sweeney wrote: src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java, lines 72-76 https://reviews.apache.org/r/34501/diff/1/?file=965743#file965743line72 How is the db storage going to graduate to production if it's not actually used when the beta flag is present? My preference is that we use whichever task store system is selected by the command-line in all codepaths - if backup recovery fails here the cluster admin can relaunch the scheduler with the H2 database disabled. Kevin Sweeney wrote: A better solution here IMO is to inject a TemporaryStorageFactory rather than call the static `DbUtil.createStorage()` factory method directly. This allows us to configure the choice of task store implementation in one place, rather than spread out across multiple packages, which appears to have contributed to this issue. Maxim Khutornenko wrote: We have to provide a solid/guaranteed way to recover from failure. Chances are very high we are in recover because of the TaskStore messing up our data. Relying on a DB task store is not a solution we can rely on in such cases. In fact, the reason I filed this bug was exactly that - I was not able to load an external backup due to DB task store violating some schema constraints. How is the db storage going to graduate to production if it's not actually used when the beta flag is present? The graduation assumes removing the in-memory task store, right? At that moment we will switch to a DB-based binding, which will become a default store. Kevin Sweeney wrote: Why is putting it behind a flag not sufficient? With your proposal we can't actually test that DbStorage works for backup recovery until we make a release that supports only DbStorage. If recovery fails with DbStorage because you have the `-enable_beta_storage` flag turned up, then it seems completely reasonable to turn that flag down and try again. Maxim Khutornenko wrote: Why is putting it behind a flag not sufficient? If DB task store messes up data at the logical rather than schema level we will not be able to guarantee the recovery until it could be too late to notice the problem. With your proposal we can't actually test that DbStorage works for backup recovery until we make a release that supports only DbStorage. My point is that we don't have to. Once the beta DB store is ready to graduate, the task store is going to be good enough to be used anywhere (including TemporaryStorage). BTW, we will still have to use a different binding to make sure the db instance is named something other than aurora. Kevin Sweeney wrote: I'm not sure I understand your concern regarding the point about using a factory binding. Could inject a `@Temporary ProviderStorage` and implement a `@Provides` method that does exactly what you describe. The code that wants the production storage instance would still consume the `Singleton`-scoped `Storage`. The implementation can be, essentially, `DbUtil.createStorage()` that respects the beta flag. Regarding the concern about messed up data at the logical level that seems like a (hopefully unlikely) risk of running with something tagged beta turned up. Disabling that codepath entirely would have prevented you from discovering this bug at all. Since it seems like we've reached a deadlock on that point, would you mind adding another committer to the people line for another opinion? I am still uneasy about defaulting to a flagged storage for recovery but I am going to accept your suggestion to unblock this diff as there is still a way to use in-memory storage. I am going to forgo the injection suggestion though as it would create yet another way to initialize DB storage, which is already way too cluttered. Refactored testModule() use cases to reduce the call pattern variety. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34501/#review84601 --- On May 20, 2015, 9:47 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34501/ --- (Updated May 20, 2015, 9:47 p.m.) Review request for Aurora and Kevin Sweeney. Bugs: AURORA-1322 https://issues.apache.org/jira/browse/AURORA-1322 Repository: aurora Description --- Defaulting TemporaryStorage to in-memory task store. Diffs - src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java 23c0c1e73a183be748199610ddf03e5d654fef74
Re: Review Request 34501: Enabling TemporaryStorage to use flagged task store.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34501/ --- (Updated May 23, 2015, 12:31 a.m.) Review request for Aurora and Kevin Sweeney. Changes --- Kevin's comments. Summary (updated) - Enabling TemporaryStorage to use flagged task store. Bugs: AURORA-1322 https://issues.apache.org/jira/browse/AURORA-1322 Repository: aurora Description (updated) --- Allowing TemporaryStorage to use command line controlled task store. Diffs (updated) - src/jmh/java/org/apache/aurora/benchmark/SnapshotBenchmarks.java 03dd5d99b269a1add709504b21fc197fdef9e18c src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java f361caa6f4e3b28f4a1e4d57f1989a6aa6fe258e src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java 23c0c1e73a183be748199610ddf03e5d654fef74 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 436d3841b9361df4db98a2217e61abb95e6e6bab src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java fe8e3f8f164732769fa0ae50a62e89c8aa77e9a5 src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java 177d720b59ba601d59aada9650aba799babb9a73 src/test/java/org/apache/aurora/scheduler/storage/mem/InMemTaskStoreTest.java 999d5e893897dc24659ca4a240ad3d4615cac0e5 Diff: https://reviews.apache.org/r/34501/diff/ Testing --- ./gradlew -Pq build Manual restore from backup in Vagrant. Thanks, Maxim Khutornenko
Re: Review Request 34440: Implementing task reconciler.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34440/ --- (Updated May 22, 2015, 8:47 p.m.) Review request for Aurora, Ben Mahler, Joshua Cohen, and Zameer Manji. Changes --- -kevints +jcohen CR comments. Bugs: AURORA-1047 https://issues.apache.org/jira/browse/AURORA-1047 Repository: aurora Description --- Adding a new service to request explict/implicit task reconciliations on a periodic basis. The reconciler is OFF by default until the GC executor code is removed. Diffs (updated) - src/jmh/java/org/apache/aurora/benchmark/fakes/FakeDriver.java 316ab1c06ce63c9a3f7232264d30a40f487fc45c src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java e9d47fda3355786a4e68eac5c28490c04bc68cb3 src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/Driver.java 975ea02b45488608286e743888de25862cc77add src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 35cada6160af01c13362fa7c14b9ff8da034 src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java 02e87989a17d95d36e61ffcef2e86c91774972bc src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 2beea4fc1a24c0050898077ecdf6cac2b19fab31 Diff: https://reviews.apache.org/r/34440/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
