Re: Review Request 35901: Add a DbCronJobStore implementation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/#review89798 --- Ship it! Thanks for your explanations, really helped. - Maxim Khutornenko On June 26, 2015, 2:13 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/ --- (Updated June 26, 2015, 2:13 a.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-415 https://issues.apache.org/jira/browse/AURORA-415 Repository: aurora Description --- This pulled a longer thread than i had hoped, but rest assured that the diff is exaggerated due to some code moves. I'll leave comments in specific places of note. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/CronJobMapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbCronJobStore.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 8bfb076a80face2703045800cf5530f37d64269f src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java bb61542f4043847b1c8c92ff1b4a0ecfcfc88973 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java a5acb3ae2ca5ecfd7d470fcd02de6091d66a1694 src/main/java/org/apache/aurora/scheduler/storage/db/InsertResult.java 44dc8f5e3dcc91e80a03d980c5d8ae0db65c8b89 src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java 46fa940b9cdf0544fa33b76e87ad21045e8bcdbb src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/CronCollisionPolicyTypeHandler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 0a519be65f90cb730f6bda1e6d7b019f0f15252b src/main/java/org/apache/aurora/scheduler/storage/db/views/CronJobWrapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/ScheduledTaskWrapper.java b89e7b5463cdb9ff9d1f9106dda0c0c4908225ca src/main/java/org/apache/aurora/scheduler/storage/mem/InMemStoresModule.java 35c83b9b8838a00693c8ebc96e496886ca249ed1 src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobStore.java f9e9e89a52c9bce3dd7e5a727498a7c87c26a56c src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobKeyMapper.xml 80ff3472c768cd116770e887b68e70d2ea3c9a8d src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml e9660b732bdc97117ee467fc124e5a014f176c67 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 0f77db7397866c008d32ea6f55c5bb577fe6468f src/test/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJobTest.java b9e16578b27de2985d24c25aae507b3540fcd3ff src/test/java/org/apache/aurora/scheduler/cron/quartz/CronIT.java 863e9c998c97506759a5526135a33a161a8fb75f src/test/java/org/apache/aurora/scheduler/cron/quartz/QuartzTestUtil.java 2d74b326831fbe22fa6045610ca8d714cd64779e src/test/java/org/apache/aurora/scheduler/storage/AbstractCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 5ad0de7c6a648f5eb6408eea7bcd789c25d55f88 src/test/java/org/apache/aurora/scheduler/storage/db/DbCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollectorTest.java 31feaea7ba74350fc199333a2384419ec05f1816 src/test/java/org/apache/aurora/scheduler/storage/mem/InMemTaskStoreTest.java 2ed748383f269217860b80c831a3521facba83ba src/test/java/org/apache/aurora/scheduler/storage/mem/MemCronJobStoreTest.java 58256afafd12e5de234755969605861891af4daf Diff: https://reviews.apache.org/r/35901/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 35901: Add a DbCronJobStore implementation.
On June 26, 2015, 2:22 a.m., Bill Farner wrote: src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml, line 26 https://reviews.apache.org/r/35901/diff/1/?file=992819#file992819line26 We don't actually fill an unset cronCollisionPolicy with the default, but handle `null` in the consuming code. I opted to maintain this behavior. This is quite ugly. Mind leaving a TODO to reconsider? - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/#review89464 --- On June 26, 2015, 2:13 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/ --- (Updated June 26, 2015, 2:13 a.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-415 https://issues.apache.org/jira/browse/AURORA-415 Repository: aurora Description --- This pulled a longer thread than i had hoped, but rest assured that the diff is exaggerated due to some code moves. I'll leave comments in specific places of note. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/CronJobMapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbCronJobStore.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 8bfb076a80face2703045800cf5530f37d64269f src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java bb61542f4043847b1c8c92ff1b4a0ecfcfc88973 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java a5acb3ae2ca5ecfd7d470fcd02de6091d66a1694 src/main/java/org/apache/aurora/scheduler/storage/db/InsertResult.java 44dc8f5e3dcc91e80a03d980c5d8ae0db65c8b89 src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java 46fa940b9cdf0544fa33b76e87ad21045e8bcdbb src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/CronCollisionPolicyTypeHandler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 0a519be65f90cb730f6bda1e6d7b019f0f15252b src/main/java/org/apache/aurora/scheduler/storage/db/views/CronJobWrapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/ScheduledTaskWrapper.java b89e7b5463cdb9ff9d1f9106dda0c0c4908225ca src/main/java/org/apache/aurora/scheduler/storage/mem/InMemStoresModule.java 35c83b9b8838a00693c8ebc96e496886ca249ed1 src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobStore.java f9e9e89a52c9bce3dd7e5a727498a7c87c26a56c src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobKeyMapper.xml 80ff3472c768cd116770e887b68e70d2ea3c9a8d src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml e9660b732bdc97117ee467fc124e5a014f176c67 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 0f77db7397866c008d32ea6f55c5bb577fe6468f src/test/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJobTest.java b9e16578b27de2985d24c25aae507b3540fcd3ff src/test/java/org/apache/aurora/scheduler/cron/quartz/CronIT.java 863e9c998c97506759a5526135a33a161a8fb75f src/test/java/org/apache/aurora/scheduler/cron/quartz/QuartzTestUtil.java 2d74b326831fbe22fa6045610ca8d714cd64779e src/test/java/org/apache/aurora/scheduler/storage/AbstractCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 5ad0de7c6a648f5eb6408eea7bcd789c25d55f88 src/test/java/org/apache/aurora/scheduler/storage/db/DbCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollectorTest.java 31feaea7ba74350fc199333a2384419ec05f1816 src/test/java/org/apache/aurora/scheduler/storage/mem/InMemTaskStoreTest.java 2ed748383f269217860b80c831a3521facba83ba src/test/java/org/apache/aurora/scheduler/storage/mem/MemCronJobStoreTest.java 58256afafd12e5de234755969605861891af4daf Diff: https://reviews.apache.org/r/35901/diff/ Testing --- Thanks, Bill Farner
Review Request 35901: Add a DbCronJobStore implementation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/ --- Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-415 https://issues.apache.org/jira/browse/AURORA-415 Repository: aurora Description --- This pulled a longer thread than i had hoped, but rest assured that the diff is exaggerated due to some code moves. I'll leave comments in specific places of note. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/CronJobMapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbCronJobStore.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 8bfb076a80face2703045800cf5530f37d64269f src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java bb61542f4043847b1c8c92ff1b4a0ecfcfc88973 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java a5acb3ae2ca5ecfd7d470fcd02de6091d66a1694 src/main/java/org/apache/aurora/scheduler/storage/db/InsertResult.java 44dc8f5e3dcc91e80a03d980c5d8ae0db65c8b89 src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java 46fa940b9cdf0544fa33b76e87ad21045e8bcdbb src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/CronCollisionPolicyTypeHandler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 0a519be65f90cb730f6bda1e6d7b019f0f15252b src/main/java/org/apache/aurora/scheduler/storage/db/views/CronJobWrapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/ScheduledTaskWrapper.java b89e7b5463cdb9ff9d1f9106dda0c0c4908225ca src/main/java/org/apache/aurora/scheduler/storage/mem/InMemStoresModule.java 35c83b9b8838a00693c8ebc96e496886ca249ed1 src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobStore.java f9e9e89a52c9bce3dd7e5a727498a7c87c26a56c src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobKeyMapper.xml 80ff3472c768cd116770e887b68e70d2ea3c9a8d src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml e9660b732bdc97117ee467fc124e5a014f176c67 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 0f77db7397866c008d32ea6f55c5bb577fe6468f src/test/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJobTest.java b9e16578b27de2985d24c25aae507b3540fcd3ff src/test/java/org/apache/aurora/scheduler/cron/quartz/CronIT.java 863e9c998c97506759a5526135a33a161a8fb75f src/test/java/org/apache/aurora/scheduler/cron/quartz/QuartzTestUtil.java 2d74b326831fbe22fa6045610ca8d714cd64779e src/test/java/org/apache/aurora/scheduler/storage/AbstractCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 5ad0de7c6a648f5eb6408eea7bcd789c25d55f88 src/test/java/org/apache/aurora/scheduler/storage/db/DbCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollectorTest.java 31feaea7ba74350fc199333a2384419ec05f1816 src/test/java/org/apache/aurora/scheduler/storage/mem/InMemTaskStoreTest.java 2ed748383f269217860b80c831a3521facba83ba src/test/java/org/apache/aurora/scheduler/storage/mem/MemCronJobStoreTest.java 58256afafd12e5de234755969605861891af4daf Diff: https://reviews.apache.org/r/35901/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 35901: Add a DbCronJobStore implementation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/#review89464 --- Adding reviewer notes. src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java https://reviews.apache.org/r/35901/#comment142098 This check is made superfluous now that `ScheduledTaskWrapper` extends `InsertResult`. src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java https://reviews.apache.org/r/35901/#comment142099 This moved to `TaskConfigManager`. src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java https://reviews.apache.org/r/35901/#comment142100 Also moved to `TaskConfigManager`. src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml (line 26) https://reviews.apache.org/r/35901/#comment142101 We don't actually fill an unset cronCollisionPolicy with the default, but handle `null` in the consuming code. I opted to maintain this behavior. src/test/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJobTest.java (line 72) https://reviews.apache.org/r/35901/#comment142102 This test validated bad data coming from storage, which is no longer possible. src/test/java/org/apache/aurora/scheduler/storage/AbstractCronJobStoreTest.java (line 43) https://reviews.apache.org/r/35901/#comment142103 The test cases in here are unchanged from the original, but the delta in the file causes git to not track it as a move. src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java (line 49) https://reviews.apache.org/r/35901/#comment142104 This test case relates to AURORA-749 [1]. While we are not ready to remove the `Identity` struct as that ticket calls for, we are now at a release where we can expect `JobConfiguration.taskConfig.job` to be set. I opted to change behavior here to avoid an if/else in the mapper code. [1] https://issues.apache.org/jira/browse/AURORA-749 - Bill Farner On June 26, 2015, 2:13 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/ --- (Updated June 26, 2015, 2:13 a.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-415 https://issues.apache.org/jira/browse/AURORA-415 Repository: aurora Description --- This pulled a longer thread than i had hoped, but rest assured that the diff is exaggerated due to some code moves. I'll leave comments in specific places of note. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/CronJobMapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbCronJobStore.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 8bfb076a80face2703045800cf5530f37d64269f src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java bb61542f4043847b1c8c92ff1b4a0ecfcfc88973 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java a5acb3ae2ca5ecfd7d470fcd02de6091d66a1694 src/main/java/org/apache/aurora/scheduler/storage/db/InsertResult.java 44dc8f5e3dcc91e80a03d980c5d8ae0db65c8b89 src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java 46fa940b9cdf0544fa33b76e87ad21045e8bcdbb src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/CronCollisionPolicyTypeHandler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 0a519be65f90cb730f6bda1e6d7b019f0f15252b src/main/java/org/apache/aurora/scheduler/storage/db/views/CronJobWrapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/ScheduledTaskWrapper.java b89e7b5463cdb9ff9d1f9106dda0c0c4908225ca src/main/java/org/apache/aurora/scheduler/storage/mem/InMemStoresModule.java 35c83b9b8838a00693c8ebc96e496886ca249ed1 src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobStore.java f9e9e89a52c9bce3dd7e5a727498a7c87c26a56c src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobKeyMapper.xml 80ff3472c768cd116770e887b68e70d2ea3c9a8d src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml e9660b732bdc97117ee467fc124e5a014f176c67 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 0f77db7397866c008d32ea6f55c5bb577fe6468f src/test/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJobTest.java b9e16578b27de2985d24c25aae507b3540fcd3ff src/test/java/org/apache/aurora/scheduler/cron/quartz/CronIT.java 863e9c998c97506759a5526135a33a161a8fb75f
Re: Review Request 35901: Add a DbCronJobStore implementation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/#review89466 --- Ship it! Master (56bb1e6) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On June 26, 2015, 2:13 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35901/ --- (Updated June 26, 2015, 2:13 a.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-415 https://issues.apache.org/jira/browse/AURORA-415 Repository: aurora Description --- This pulled a longer thread than i had hoped, but rest assured that the diff is exaggerated due to some code moves. I'll leave comments in specific places of note. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/CronJobMapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbCronJobStore.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 8bfb076a80face2703045800cf5530f37d64269f src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java bb61542f4043847b1c8c92ff1b4a0ecfcfc88973 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java a5acb3ae2ca5ecfd7d470fcd02de6091d66a1694 src/main/java/org/apache/aurora/scheduler/storage/db/InsertResult.java 44dc8f5e3dcc91e80a03d980c5d8ae0db65c8b89 src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java 46fa940b9cdf0544fa33b76e87ad21045e8bcdbb src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/CronCollisionPolicyTypeHandler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 0a519be65f90cb730f6bda1e6d7b019f0f15252b src/main/java/org/apache/aurora/scheduler/storage/db/views/CronJobWrapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/ScheduledTaskWrapper.java b89e7b5463cdb9ff9d1f9106dda0c0c4908225ca src/main/java/org/apache/aurora/scheduler/storage/mem/InMemStoresModule.java 35c83b9b8838a00693c8ebc96e496886ca249ed1 src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobStore.java f9e9e89a52c9bce3dd7e5a727498a7c87c26a56c src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobKeyMapper.xml 80ff3472c768cd116770e887b68e70d2ea3c9a8d src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml e9660b732bdc97117ee467fc124e5a014f176c67 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 0f77db7397866c008d32ea6f55c5bb577fe6468f src/test/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJobTest.java b9e16578b27de2985d24c25aae507b3540fcd3ff src/test/java/org/apache/aurora/scheduler/cron/quartz/CronIT.java 863e9c998c97506759a5526135a33a161a8fb75f src/test/java/org/apache/aurora/scheduler/cron/quartz/QuartzTestUtil.java 2d74b326831fbe22fa6045610ca8d714cd64779e src/test/java/org/apache/aurora/scheduler/storage/AbstractCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 5ad0de7c6a648f5eb6408eea7bcd789c25d55f88 src/test/java/org/apache/aurora/scheduler/storage/db/DbCronJobStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollectorTest.java 31feaea7ba74350fc199333a2384419ec05f1816 src/test/java/org/apache/aurora/scheduler/storage/mem/InMemTaskStoreTest.java 2ed748383f269217860b80c831a3521facba83ba src/test/java/org/apache/aurora/scheduler/storage/mem/MemCronJobStoreTest.java 58256afafd12e5de234755969605861891af4daf Diff: https://reviews.apache.org/r/35901/diff/ Testing --- Thanks, Bill Farner
