Re: Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
On Aug. 4, 2015, 6:57 p.m., Maxim Khutornenko wrote: src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java, line 154 https://reviews.apache.org/r/37049/diff/1/?file=1028142#file1028142line154 This seems like a new pattern that's inconsistent with the ScheduledExecutorService interface. Should the DelayExecutor take long and TimeUnit separately instead? Bill Farner wrote: What's the upside? This self-documents just fine, and results in a net removal of code for us. Just consistency with the parent interface as we now have cases where we shred Amount to pieces to satisfy ScheduledExecutorService. Up to you. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/#review94101 --- On Aug. 4, 2015, 5:17 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- (Updated Aug. 4, 2015, 5:17 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs - src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java e198c1caca201d2a599404341d3ada6fae8f1b9d Diff: https://reviews.apache.org/r/37049/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- (Updated Aug. 5, 2015, 6:51 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Changes --- Fixed benchmarks. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs (updated) - src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java d75f0901ff88995467c07c49630252b7868802b9 src/jmh/java/org/apache/aurora/benchmark/ThriftApiBenchmarks.java b2a3e9bdf5ed42c2b5d71a1f677be988525b2f1d src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java e198c1caca201d2a599404341d3ada6fae8f1b9d Diff: https://reviews.apache.org/r/37049/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/#review94282 --- src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java https://reviews.apache.org/r/37049/#comment148835 Are you certain this is no longer needed? The setup above is shared accross iterations making it possible for the queue to grow unbounded. - Maxim Khutornenko On Aug. 5, 2015, 6:51 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- (Updated Aug. 5, 2015, 6:51 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs - src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java d75f0901ff88995467c07c49630252b7868802b9 src/jmh/java/org/apache/aurora/benchmark/ThriftApiBenchmarks.java b2a3e9bdf5ed42c2b5d71a1f677be988525b2f1d src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java e198c1caca201d2a599404341d3ada6fae8f1b9d Diff: https://reviews.apache.org/r/37049/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/#review94283 --- Ship it! Master (2130f7e) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On Aug. 5, 2015, 6:51 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- (Updated Aug. 5, 2015, 6:51 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs - src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java d75f0901ff88995467c07c49630252b7868802b9 src/jmh/java/org/apache/aurora/benchmark/ThriftApiBenchmarks.java b2a3e9bdf5ed42c2b5d71a1f677be988525b2f1d src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java e198c1caca201d2a599404341d3ada6fae8f1b9d Diff: https://reviews.apache.org/r/37049/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
On Aug. 5, 2015, 7:03 p.m., Maxim Khutornenko wrote: src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, lines 192-197 https://reviews.apache.org/r/37049/diff/2/?file=1032776#file1032776line192 Are you certain this is no longer needed? The setup above is shared accross iterations making it possible for the queue to grow unbounded. Bill Farner wrote: There's no queue any more, so there's nothing to clear or wait for. Indeed. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/#review94282 --- On Aug. 5, 2015, 6:51 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- (Updated Aug. 5, 2015, 6:51 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs - src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java d75f0901ff88995467c07c49630252b7868802b9 src/jmh/java/org/apache/aurora/benchmark/ThriftApiBenchmarks.java b2a3e9bdf5ed42c2b5d71a1f677be988525b2f1d src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java e198c1caca201d2a599404341d3ada6fae8f1b9d Diff: https://reviews.apache.org/r/37049/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/#review94101 --- LGTM. A few comments/questions below. src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java (line 86) https://reviews.apache.org/r/37049/#comment148597 Don't you want in(Singleton.class) here? src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java (line 152) https://reviews.apache.org/r/37049/#comment148598 This seems like a new pattern that's inconsistent with the ScheduledExecutorService interface. Should the DelayExecutor take long and TimeUnit separately instead? src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java (line 159) https://reviews.apache.org/r/37049/#comment148606 Will SnapshotBenchmarks still work with this binding? src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java (line 154) https://reviews.apache.org/r/37049/#comment148603 How about converting this to a TODO and/or a ticket to follow up with cancellation revision? Feels like transaction abortion/rollback ideally should not result in any side-effects. - Maxim Khutornenko On Aug. 4, 2015, 5:17 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- (Updated Aug. 4, 2015, 5:17 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs - src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java
Re: Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/#review93983 --- Ship it! Master (7fdcfef) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On Aug. 3, 2015, 11:49 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- (Updated Aug. 3, 2015, 11:49 p.m.) Review request for Aurora and Kevin Sweeney. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs - src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java e198c1caca201d2a599404341d3ada6fae8f1b9d Diff: https://reviews.apache.org/r/37049/diff/ Testing --- Thanks, Bill Farner
Review Request 37049: Integrating DelayExecutor into the scheduler's transaction handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37049/ --- Review request for Aurora and Kevin Sweeney. Bugs: AURORA-1395 https://issues.apache.org/jira/browse/AURORA-1395 Repository: aurora Description --- This creates two fundamental ways of performing asynchronous work in the scheduler: - Consume `@AsyncExecutor DelayExecutor` to perform work in the context of a database transaction, completing the work after the transaction - Consume `@BackgroundWorker ScheduledExecutorService` to perform work entirely in the background Diffs - src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java c345c926512431e447649689d152b1ffa876ee85 src/main/java/org/apache/aurora/scheduler/async/GatedDelayExecutor.java 1893a9bcbf6167177d8d0e83ec8ea4c10da899f0 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 4b8a55fe5530c0b3e483838035cd1ee0f00a2c71 src/main/java/org/apache/aurora/scheduler/pruning/TaskHistoryPruner.java fa9a09c19ad56cb0c13460464c6445cfe326c89b src/main/java/org/apache/aurora/scheduler/reconciliation/KillRetry.java 1611a3b69776ef72ed29fa2bd73df26442183a26 src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java 406c07767b2bd4f4d80b9f0af72ae1cb4e457bc0 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskReconciler.java 653e52b340edd6c87c12b0fdf760edf6d9d62722 src/main/java/org/apache/aurora/scheduler/reconciliation/TaskTimeout.java fb8397260406cd45fcdfec17872b6ad4a011f2c5 src/main/java/org/apache/aurora/scheduler/scheduling/TaskGroups.java e60daad8606a7060f690f9a75d9c5bce0107522c src/main/java/org/apache/aurora/scheduler/scheduling/TaskThrottler.java e54e6c4b5b43b604110da11680b002f6d4b12df9 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ed92661605e27673c176b91535e4773be2a920a3 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java a1f0d3ca89189a7791e823fec708b41a164cb7e0 src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970 src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 5384307a6e4bf27800ab9e5104d0e74ac748e515 src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 91b91bc7bbaeaa5341565a8ee1ac3a97ea8a1620 src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java e725e102c4321c3aa30d89581783744475897f94 src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 088a4a6e874acc4926414bc30806e45fb961adec src/test/java/org/apache/aurora/scheduler/pruning/TaskHistoryPrunerTest.java 532b0ea70504b8b13f5e0ada5a519635c804a4b7 src/test/java/org/apache/aurora/scheduler/reconciliation/KillRetryTest.java 26f65fa99eab0a4d56d3e2da16e5704223c53780 src/test/java/org/apache/aurora/scheduler/reconciliation/TaskTimeoutTest.java 97d25f9b20e49783e1576910883201f6e37f584f src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java 55aad3572e3b93a73085dc94b135b63b81454cbc src/test/java/org/apache/aurora/scheduler/scheduling/TaskThrottlerTest.java 405502122325927aa90b36a89c55667042e01ca7 src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java a4bcdd7ad06da21e17c43a53854658049e03eb0e src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java e198c1caca201d2a599404341d3ada6fae8f1b9d Diff: https://reviews.apache.org/r/37049/diff/ Testing --- Thanks, Bill Farner
