Re: Review Request 25812: Implementing quota checking for async job updates.
On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java, line 268 https://reviews.apache.org/r/25812/diff/1/?file=694286#file694286line268 Remember - javadoc is like html. These newlines are not preserved unless you do it explicitly. Drop a p here, ditto in other comments. This was mostly for in-code readability but adding p would not hurt. On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java, line 274 https://reviews.apache.org/r/25812/diff/1/?file=694286#file694286line274 how about s/prodFromTasks/getRequiredQuota/? Well, quota here means one thing - pre-defined cap to compare against. I feel it would be too confusing to overload that term for consumption. Changed it to prodResourcesFromTasks instead. On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, line 354 https://reviews.apache.org/r/25812/diff/1/?file=694289#file694289line354 Catching IllegalArgumentException is likely to be more broad than you intend. I suggest you throw a specific exception from the validate function to avoid incorrectly replying 400 for something that should be 500. Agree, thanks for pushing it. On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, line 1350 https://reviews.apache.org/r/25812/diff/1/?file=694289#file694289line1350 How about 'validateInstanceAddition'? I'm not tied to the name, but something more informative than 'validate' would be nice. How about validateTaskLimits? On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/thrift/org/apache/aurora/gen/api.thrift, line 547 https://reviews.apache.org/r/25812/diff/1/?file=694290#file694290line547 Can you update JobUpdateControllerImpl to use this as well? Also, you should wrap this with a constant that we define. Thrift produces a mutable static Set for this, which comes with risk. The only place it's used in is already wrapping it into EnumSet. What specifically in JobUpdateControllerImpl? Could not find anything suitable to update. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54168 --- On Sept. 23, 2014, 12:59 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 12:59 a.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 779e925e4d9e7889e8cfd369cea9a8e5da3554d2 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 83ac034cff009530e5e16c0613b9d085f3b908d8 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 49770e5f87f047502e4f5653b908657a40d8683f src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 336cada625b85618486660fc24f3e83a312609f8 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks,
Re: Review Request 25812: Implementing quota checking for async job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 4:12 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Changes --- CR comments. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, line 1350 https://reviews.apache.org/r/25812/diff/1/?file=694289#file694289line1350 How about 'validateInstanceAddition'? I'm not tied to the name, but something more informative than 'validate' would be nice. Maxim Khutornenko wrote: How about validateTaskLimits? Sounds good. On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/thrift/org/apache/aurora/gen/api.thrift, line 547 https://reviews.apache.org/r/25812/diff/1/?file=694290#file694290line547 Can you update JobUpdateControllerImpl to use this as well? Also, you should wrap this with a constant that we define. Thrift produces a mutable static Set for this, which comes with risk. Maxim Khutornenko wrote: The only place it's used in is already wrapping it into EnumSet. What specifically in JobUpdateControllerImpl? Could not find anything suitable to update. https://github.com/apache/incubator-aurora/blob/012aca3d2a5667f9cfd6a1b9b0b91c982d34bb94/src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java#L254-L258 - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54168 --- On Sept. 23, 2014, 4:12 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 4:12 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
On Sept. 23, 2014, 1:01 a.m., Bill Farner wrote: src/main/thrift/org/apache/aurora/gen/api.thrift, line 547 https://reviews.apache.org/r/25812/diff/1/?file=694290#file694290line547 Can you update JobUpdateControllerImpl to use this as well? Also, you should wrap this with a constant that we define. Thrift produces a mutable static Set for this, which comes with risk. Maxim Khutornenko wrote: The only place it's used in is already wrapping it into EnumSet. What specifically in JobUpdateControllerImpl? Could not find anything suitable to update. Bill Farner wrote: https://github.com/apache/incubator-aurora/blob/012aca3d2a5667f9cfd6a1b9b0b91c982d34bb94/src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java#L254-L258 Ah, I was searching the non-rebased branch. Done. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54168 --- On Sept. 23, 2014, 4:12 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 4:12 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 4:48 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Changes --- CR comments. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25466: Upgrade Aurora to psutil 2.1.1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25466/#review54282 --- psutil 2.1.2 now available, since we are upgrading here might as well go with the latest version https://github.com/giampaolo/psutil/blob/master/HISTORY.rst - Jake Farrell On Sept. 9, 2014, 6:38 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25466/ --- (Updated Sept. 9, 2014, 6:38 p.m.) Review request for Aurora, Joe Smith and Kevin Sweeney. Bugs: AURORA-412 https://issues.apache.org/jira/browse/AURORA-412 Repository: aurora Description --- Upgrade Aurora to psutil 2.1.1 Diffs - 3rdparty/python/BUILD 5e96b91aa11567f51ed599c9d585f3af4e4e7127 src/main/python/apache/aurora/executor/executor_vars.py 7665a77ca13bbbc337e345356b70f12357692f3d src/main/python/apache/aurora/executor/gc_executor.py 572c106038602b795a5b6bf08b2b85cf4b6bfec6 src/main/python/apache/thermos/core/helper.py 9e4a8fb70df984d175129cb49025b3c222a64679 src/main/python/apache/thermos/monitoring/process_collector_psutil.py 298459bab01d2c01bef1d69c1150ff2d5737ebbf src/test/python/apache/thermos/core/test_helper.py 58096119e1b412afa36a09da5759c5223a5b5d75 Diff: https://reviews.apache.org/r/25466/diff/ Testing --- Ran tests on OS X but not yet on Linux. There is adequate coverage. ==edit, Linux test output== src.test.python.apache.thermos.bin.test_thermos . SUCCESS src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.common.test_planner . SUCCESS src.test.python.apache.thermos.common.test_task_planner . SUCCESS src.test.python.apache.thermos.config.test_schema . SUCCESS src.test.python.apache.thermos.core.test_angry . SUCCESS src.test.python.apache.thermos.core.test_ephemerals . SUCCESS src.test.python.apache.thermos.core.test_failing_runner . SUCCESS src.test.python.apache.thermos.core.test_failure_limit . SUCCESS src.test.python.apache.thermos.core.test_finalization . SUCCESS src.test.python.apache.thermos.core.test_helper . SUCCESS src.test.python.apache.thermos.core.test_process . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.core.test_staged_kill . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS src.test.python.apache.aurora.admin.admin_util . SUCCESS src.test.python.apache.aurora.admin.host_maintenance . SUCCESS src.test.python.apache.aurora.client.api.api . SUCCESS src.test.python.apache.aurora.client.api.disambiguator . SUCCESS src.test.python.apache.aurora.client.api.instance_watcher . SUCCESS src.test.python.apache.aurora.client.api.job_monitor . SUCCESS src.test.python.apache.aurora.client.api.mux . SUCCESS src.test.python.apache.aurora.client.api.quota_check . SUCCESS src.test.python.apache.aurora.client.api.restarter . SUCCESS src.test.python.apache.aurora.client.api.scheduler_client . SUCCESS src.test.python.apache.aurora.client.api.sla . SUCCESS src.test.python.apache.aurora.client.api.updater . SUCCESS src.test.python.apache.aurora.client.api.updater_util . SUCCESS src.test.python.apache.aurora.client.binding_helper . SUCCESS src.test.python.apache.aurora.client.cli.api . SUCCESS src.test.python.apache.aurora.client.cli.bridge . SUCCESS src.test.python.apache.aurora.client.cli.command_hooks . SUCCESS src.test.python.apache.aurora.client.cli.config . SUCCESS
Re: Review Request 25812: Implementing quota checking for async job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54315 --- src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java https://reviews.apache.org/r/25812/#comment94426 Unless I'm missing something here, this will throw when there are multiple updates for a single role? I think you want Multimaps.index here. src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java https://reviews.apache.org/r/25812/#comment94419 Refactor this to live outside the Impl class? SchedulerThriftInterface shouldn't need about implementation classes - Kevin Sweeney On Sept. 23, 2014, 9:48 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 9:48 a.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54316 --- LGTM with some additional test coverage for the new accounting. I think that will help make the desired output of the algorithm easier to understand in the future. src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java https://reviews.apache.org/r/25812/#comment94420 Accept JobUpdateStore here, that's all that is used. src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java https://reviews.apache.org/r/25812/#comment94422 Can you leave a longer comment here? I understand what is going on, but i'm not sure other readers will have the context to follow the algorithm. src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java https://reviews.apache.org/r/25812/#comment94423 Can you extract this predicate to a factory function? I think it might aid readability. src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java https://reviews.apache.org/r/25812/#comment94424 space between * and p src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java https://reviews.apache.org/r/25812/#comment94425 space between * and p src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java https://reviews.apache.org/r/25812/#comment94428 It would be great if these test cases validated the actual computed consumption here. Can you also call quotaManager.getQuotaInfo to validate the accounting? src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java https://reviews.apache.org/r/25812/#comment94427 Can you add 2 cases: - update adds instances - update removes instances - Bill Farner On Sept. 23, 2014, 4:48 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 4:48 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Review Request 25953: Allow optional initial and desired states.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25953/ --- Review request for Aurora and Bill Farner. Bugs: AURORA-741 https://issues.apache.org/jira/browse/AURORA-741 Repository: aurora Description --- Added safety checks around initial/desired state presence and converted to left outer joins. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java 361d3b8c0d59a0dad6b00c8a4b78ec8284383586 src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 80535432c2c45eb2f3eb3bea4a797a99ea4fc2c9 src/test/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStoreTest.java aa7feb916f879f60d46ebaf53e47ba30d861c7fa Diff: https://reviews.apache.org/r/25953/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25953: Allow optional initial and desired states.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25953/#review54328 --- Ship it! LGTM with a few minor changes. src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java https://reviews.apache.org/r/25953/#comment94441 You should include a precondition check for desiredState being set or initialState being non-empty. src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java https://reviews.apache.org/r/25953/#comment94443 !isEmpty() - Bill Farner On Sept. 23, 2014, 8:38 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25953/ --- (Updated Sept. 23, 2014, 8:38 p.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-741 https://issues.apache.org/jira/browse/AURORA-741 Repository: aurora Description --- Added safety checks around initial/desired state presence and converted to left outer joins. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java 361d3b8c0d59a0dad6b00c8a4b78ec8284383586 src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 80535432c2c45eb2f3eb3bea4a797a99ea4fc2c9 src/test/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStoreTest.java aa7feb916f879f60d46ebaf53e47ba30d861c7fa Diff: https://reviews.apache.org/r/25953/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25953: Allow optional initial and desired states.
On Sept. 23, 2014, 8:48 p.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java, line 87 https://reviews.apache.org/r/25953/diff/1/?file=703165#file703165line87 You should include a precondition check for desiredState being set or initialState being non-empty. Sure, added. On Sept. 23, 2014, 8:48 p.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java, line 103 https://reviews.apache.org/r/25953/diff/1/?file=703165#file703165line103 !isEmpty() Done. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25953/#review54328 --- On Sept. 23, 2014, 8:38 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25953/ --- (Updated Sept. 23, 2014, 8:38 p.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-741 https://issues.apache.org/jira/browse/AURORA-741 Repository: aurora Description --- Added safety checks around initial/desired state presence and converted to left outer joins. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java 361d3b8c0d59a0dad6b00c8a4b78ec8284383586 src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 80535432c2c45eb2f3eb3bea4a797a99ea4fc2c9 src/test/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStoreTest.java aa7feb916f879f60d46ebaf53e47ba30d861c7fa Diff: https://reviews.apache.org/r/25953/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25953: Allow optional initial and desired states.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25953/ --- (Updated Sept. 23, 2014, 9:10 p.m.) Review request for Aurora and Bill Farner. Changes --- CR comments. Bugs: AURORA-741 https://issues.apache.org/jira/browse/AURORA-741 Repository: aurora Description --- Added safety checks around initial/desired state presence and converted to left outer joins. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java 361d3b8c0d59a0dad6b00c8a4b78ec8284383586 src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 80535432c2c45eb2f3eb3bea4a797a99ea4fc2c9 src/test/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStoreTest.java aa7feb916f879f60d46ebaf53e47ba30d861c7fa Diff: https://reviews.apache.org/r/25953/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
On Sept. 23, 2014, 1:07 p.m., Kevin Sweeney wrote: src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java, line 180 https://reviews.apache.org/r/25812/diff/3/?file=702913#file702913line180 Unless I'm missing something here, this will throw when there are multiple updates for a single role? I think you want Multimaps.index here. Bill Farner wrote: Multiple _active_ updates are not allowed. ah, my mistake - Kevin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54315 --- On Sept. 23, 2014, 9:48 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 9:48 a.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54334 --- Ship it! LGTM mod Bill's comments and Impl suggestion - Kevin Sweeney On Sept. 23, 2014, 9:48 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 9:48 a.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Review Request 25955: Remove about plugin.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25955/ --- Review request for Aurora and David McLaughlin. Repository: aurora Description --- I've spent way too much time waiting for this plugin to complete, and it has a tendency to fail sporadically (especially when running a distribution not in a git repo). Since we don't really use the output anyway, i'm fine to drop it. Diffs - build.gradle e51912116c2a205bb85f229b167ddb3e1b3b7883 Diff: https://reviews.apache.org/r/25955/diff/ Testing --- ./gradlew build -Pq Thanks, Bill Farner
Re: Review Request 25955: Remove about plugin.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25955/#review54337 --- Ship it! Ship It! - David McLaughlin On Sept. 23, 2014, 9:35 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25955/ --- (Updated Sept. 23, 2014, 9:35 p.m.) Review request for Aurora and David McLaughlin. Repository: aurora Description --- I've spent way too much time waiting for this plugin to complete, and it has a tendency to fail sporadically (especially when running a distribution not in a git repo). Since we don't really use the output anyway, i'm fine to drop it. Diffs - build.gradle e51912116c2a205bb85f229b167ddb3e1b3b7883 Diff: https://reviews.apache.org/r/25955/diff/ Testing --- ./gradlew build -Pq Thanks, Bill Farner
Review Request 25958: Enable beta updater in example cluster.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25958/ --- Review request for Aurora, Maxim Khutornenko and Bill Farner. Repository: aurora Description --- enable beta updater in example cluster Diffs - examples/vagrant/upstart/aurora-scheduler.conf 7e2b3f0191cbc9f0b2ce916a1ad0b55281d718d6 Diff: https://reviews.apache.org/r/25958/diff/ Testing --- $ vagrant provision $ vagrant ssh $ aurora2 update start devcluster/www-data/devel/hello_world hello_world.aurora log(info): Starting update for: hello_world log(info): Starting new HTTP connection (1): 192.168.33.7 log(info): Starting new HTTP connection (1): 192.168.33.7 Scheduler-driven update of job devcluster/www-data/devel/hello_world has started. Thanks, David McLaughlin
Re: Review Request 25958: Enable beta updater in example cluster.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25958/#review54345 --- Ship it! Ship It! - Maxim Khutornenko On Sept. 23, 2014, 10:08 p.m., David McLaughlin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25958/ --- (Updated Sept. 23, 2014, 10:08 p.m.) Review request for Aurora, Maxim Khutornenko and Bill Farner. Repository: aurora Description --- enable beta updater in example cluster Diffs - examples/vagrant/upstart/aurora-scheduler.conf 7e2b3f0191cbc9f0b2ce916a1ad0b55281d718d6 Diff: https://reviews.apache.org/r/25958/diff/ Testing --- $ vagrant provision $ vagrant ssh $ aurora2 update start devcluster/www-data/devel/hello_world hello_world.aurora log(info): Starting update for: hello_world log(info): Starting new HTTP connection (1): 192.168.33.7 log(info): Starting new HTTP connection (1): 192.168.33.7 Scheduler-driven update of job devcluster/www-data/devel/hello_world has started. Thanks, David McLaughlin
Re: Review Request 25812: Implementing quota checking for async job updates.
On Sept. 23, 2014, 8:10 p.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java, line 174 https://reviews.apache.org/r/25812/diff/3/?file=702913#file702913line174 Accept JobUpdateStore here, that's all that is used. Done. On Sept. 23, 2014, 8:10 p.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java, line 182 https://reviews.apache.org/r/25812/diff/3/?file=702913#file702913line182 Can you leave a longer comment here? I understand what is going on, but i'm not sure other readers will have the context to follow the algorithm. Reworded. On Sept. 23, 2014, 8:10 p.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java, line 186 https://reviews.apache.org/r/25812/diff/3/?file=702913#file702913line186 Can you extract this predicate to a factory function? I think it might aid readability. Done. On Sept. 23, 2014, 8:10 p.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java, line 267 https://reviews.apache.org/r/25812/diff/3/?file=702913#file702913line267 space between * and p Done. On Sept. 23, 2014, 8:10 p.m., Bill Farner wrote: src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java, line 325 https://reviews.apache.org/r/25812/diff/3/?file=702920#file702920line325 It would be great if these test cases validated the actual computed consumption here. Can you also call quotaManager.getQuotaInfo to validate the accounting? Sure, added coverage for consumption. On Sept. 23, 2014, 8:10 p.m., Bill Farner wrote: src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java, line 403 https://reviews.apache.org/r/25812/diff/3/?file=702920#file702920line403 Can you add 2 cases: - update adds instances - update removes instances Added. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54316 --- On Sept. 23, 2014, 4:48 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 4:48 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
On Sept. 23, 2014, 8:07 p.m., Kevin Sweeney wrote: src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, line 1469 https://reviews.apache.org/r/25812/diff/3/?file=702916#file702916line1469 Refactor this to live outside the Impl class? SchedulerThriftInterface shouldn't need about implementation classes Moved to QuotaUtil. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/#review54315 --- On Sept. 23, 2014, 4:48 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 4:48 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 25812: Implementing quota checking for async job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25812/ --- (Updated Sept. 23, 2014, 11:08 p.m.) Review request for Aurora, David McLaughlin, Kevin Sweeney, and Bill Farner. Changes --- CR comments. Bugs: AURORA-686 https://issues.apache.org/jira/browse/AURORA-686 Repository: aurora Description --- Getting it right required a deep refactoring of the QuotaManager. The prod consumption is now calculated from: - production tasks not participating in job updates; - unaffected production tasks from a job being updated (i.e. those that are not covered by the update working set); - production tasks directly affected by the job update (max of old and new resources used). Also, moved all logic back to SchedulerThriftInterface as quota checks are done only there now. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 3661f8487985f631e3ea437fe6430e0296376a9e src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 14b0dd8f86026840d0444c128f656a144eab017d src/main/java/org/apache/aurora/scheduler/quota/QuotaUtil.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/state/StateModule.java 54b90127551c69509dbd41ee95c384dbbf1a7ee4 src/main/java/org/apache/aurora/scheduler/state/TaskLimitValidator.java 70333737ee119ea083f0ac17a4205e7af9f4c753 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 8381e2126c62c40d28a325c72686d01e82bb84cf src/test/java/org/apache/aurora/scheduler/state/TaskLimitValidatorTest.java 8f18617b2052201f87bb1464314c2ee45b279276 src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 5aebbfbc691dfac4a066cb1425d18d3fccc77090 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 40156c211a346664c0d2f174235efb2049cf3bb9 Diff: https://reviews.apache.org/r/25812/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Review Request 25963: Expose in progress and recently completed updates in UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/ --- Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner. Bugs: AURORA-745 https://issues.apache.org/jira/browse/AURORA-745 Repository: aurora Description --- Very simple first iteration. Shows only the latest 20 updates. Diffs - src/main/java/org/apache/aurora/scheduler/http/JettyServerModule.java 392b4f7bc9d5352ed536ff83eafb4175e4860aa1 src/main/resources/org/apache/aurora/scheduler/http/ui/css/app.css befd5908e51dcac2a0fc56b12651af11273d25c0 src/main/resources/org/apache/aurora/scheduler/http/ui/index.html 6434d1bc3b9f769f6f565d601fd98f99743efe8c src/main/resources/org/apache/aurora/scheduler/http/ui/js/app.js 082d920348f12b214b8ca2d20c11e7b4825453ed src/main/resources/org/apache/aurora/scheduler/http/ui/js/controllers.js ded175a1e4b288b8978235c986db2549c22dcc7d src/main/resources/org/apache/aurora/scheduler/http/ui/js/services.js f65cb5e69c2d1712b5f9da782ee2c7b3bc22cee8 src/main/resources/org/apache/aurora/scheduler/http/ui/updateList.html PRE-CREATION Diff: https://reviews.apache.org/r/25963/diff/ Testing --- ./gradlew jsHint File Attachments updates page https://reviews.apache.org/media/uploaded/files/2014/09/23/33b2b9ff-3485-4bba-a8e8-2d4e6280e5ca__Screen_Shot_2014-09-23_at_4.15.06_PM.png no data updates https://reviews.apache.org/media/uploaded/files/2014/09/23/5fe9c0bf-ef31-41a3-95c4-7aeb9abcb3b0__Screen_Shot_2014-09-23_at_4.45.59_PM.png Thanks, David McLaughlin
Re: Review Request 25963: Expose in progress and recently completed updates in UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/#review54363 --- I just realised, we might want to hide the nav if beta_updates_enabled is false. - David McLaughlin On Sept. 23, 2014, 11:46 p.m., David McLaughlin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/ --- (Updated Sept. 23, 2014, 11:46 p.m.) Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner. Bugs: AURORA-745 https://issues.apache.org/jira/browse/AURORA-745 Repository: aurora Description --- Very simple first iteration. Shows only the latest 20 updates. Diffs - src/main/java/org/apache/aurora/scheduler/http/JettyServerModule.java 392b4f7bc9d5352ed536ff83eafb4175e4860aa1 src/main/resources/org/apache/aurora/scheduler/http/ui/css/app.css befd5908e51dcac2a0fc56b12651af11273d25c0 src/main/resources/org/apache/aurora/scheduler/http/ui/index.html 6434d1bc3b9f769f6f565d601fd98f99743efe8c src/main/resources/org/apache/aurora/scheduler/http/ui/js/app.js 082d920348f12b214b8ca2d20c11e7b4825453ed src/main/resources/org/apache/aurora/scheduler/http/ui/js/controllers.js ded175a1e4b288b8978235c986db2549c22dcc7d src/main/resources/org/apache/aurora/scheduler/http/ui/js/services.js f65cb5e69c2d1712b5f9da782ee2c7b3bc22cee8 src/main/resources/org/apache/aurora/scheduler/http/ui/updateList.html PRE-CREATION Diff: https://reviews.apache.org/r/25963/diff/ Testing --- ./gradlew jsHint File Attachments updates page https://reviews.apache.org/media/uploaded/files/2014/09/23/33b2b9ff-3485-4bba-a8e8-2d4e6280e5ca__Screen_Shot_2014-09-23_at_4.15.06_PM.png no data updates https://reviews.apache.org/media/uploaded/files/2014/09/23/5fe9c0bf-ef31-41a3-95c4-7aeb9abcb3b0__Screen_Shot_2014-09-23_at_4.45.59_PM.png Thanks, David McLaughlin
Re: Review Request 25963: Expose in progress and recently completed updates in UI
On Sept. 23, 2014, 11:48 p.m., David McLaughlin wrote: I just realised, we might want to hide the nav if beta_updates_enabled is false. Don't worry abou that - we'll drop the flag entirely soon. Just wanted to put a gate in place for now. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/#review54363 --- On Sept. 23, 2014, 11:46 p.m., David McLaughlin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/ --- (Updated Sept. 23, 2014, 11:46 p.m.) Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner. Bugs: AURORA-745 https://issues.apache.org/jira/browse/AURORA-745 Repository: aurora Description --- Very simple first iteration. Shows only the latest 20 updates. Diffs - src/main/java/org/apache/aurora/scheduler/http/JettyServerModule.java 392b4f7bc9d5352ed536ff83eafb4175e4860aa1 src/main/resources/org/apache/aurora/scheduler/http/ui/css/app.css befd5908e51dcac2a0fc56b12651af11273d25c0 src/main/resources/org/apache/aurora/scheduler/http/ui/index.html 6434d1bc3b9f769f6f565d601fd98f99743efe8c src/main/resources/org/apache/aurora/scheduler/http/ui/js/app.js 082d920348f12b214b8ca2d20c11e7b4825453ed src/main/resources/org/apache/aurora/scheduler/http/ui/js/controllers.js ded175a1e4b288b8978235c986db2549c22dcc7d src/main/resources/org/apache/aurora/scheduler/http/ui/js/services.js f65cb5e69c2d1712b5f9da782ee2c7b3bc22cee8 src/main/resources/org/apache/aurora/scheduler/http/ui/updateList.html PRE-CREATION Diff: https://reviews.apache.org/r/25963/diff/ Testing --- ./gradlew jsHint File Attachments updates page https://reviews.apache.org/media/uploaded/files/2014/09/23/33b2b9ff-3485-4bba-a8e8-2d4e6280e5ca__Screen_Shot_2014-09-23_at_4.15.06_PM.png no data updates https://reviews.apache.org/media/uploaded/files/2014/09/23/5fe9c0bf-ef31-41a3-95c4-7aeb9abcb3b0__Screen_Shot_2014-09-23_at_4.45.59_PM.png Thanks, David McLaughlin
Review Request 25970: Adding support for retryable storage errors.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25970/ --- Review request for Aurora, Mark Chu-Carroll, Bill Farner, and Brian Wickman. Bugs: AURORA-187 https://issues.apache.org/jira/browse/AURORA-187 Repository: aurora Description --- Adding a new ERROR_RETRYABLE thrift response type to enable response-aware retrying in scheduler_client. Diffs - src/main/java/org/apache/aurora/scheduler/storage/CallOrderEnforcingStorage.java 671dbd1b14e759f02f655909857099368255f214 src/main/java/org/apache/aurora/scheduler/storage/Storage.java 775564e1de0bec7fcfd96ea5ee3ee4948567e58f src/main/java/org/apache/aurora/scheduler/thrift/aop/LoggingInterceptor.java a21ab901d86f74d5bc22bfffc09029c99d3a398f src/main/python/apache/aurora/client/api/scheduler_client.py ec632516655bbca4782e01310518843ce8aca27e src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/python/apache/aurora/client/api/test_scheduler_client.py 527448f882c12fe9bb51ce2a0a4c330188838282 Diff: https://reviews.apache.org/r/25970/diff/ Testing --- gradle -Pq build ./pants src/test/python:all Thanks, Maxim Khutornenko
Review Request 25969: When creating an update, store only the delta between the initial and desired states.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25969/ --- Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-718 https://issues.apache.org/jira/browse/AURORA-718 Repository: aurora Description --- When creating an update, store only the delta between the initial and desired states. Diffs - src/main/java/org/apache/aurora/scheduler/base/Numbers.java 5fd0814e14eee7eb827b3a3b1e6dc28ce910b7e8 src/main/java/org/apache/aurora/scheduler/base/Tasks.java 9664c3b0f04087ceb0451a4cc5a0b4dd6cdcee9d src/main/java/org/apache/aurora/scheduler/http/TransformationUtils.java b7143941feb4824aee609741e8e2c0e015eecac3 src/main/java/org/apache/aurora/scheduler/storage/ForwardingStore.java 1874a3e3f9bc47104b7597fd88d77c0c36620f7d src/main/java/org/apache/aurora/scheduler/storage/JobUpdateStore.java cb74d62878af1efe9d58e3d4c3e5f7a84edc8de6 src/main/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStore.java 15a9e2b9acd65f6268e70fb402414f5c01c42409 src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java 26cc85dcb6dccd913e9a130a134fbf8df10cc08d src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java ae0320b44b55a3630e255484ea7a881daab6a7bc src/main/java/org/apache/aurora/scheduler/updater/JobDiff.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/updater/JobUpdateController.java 8f9e6d6fa00840d8ff94eb7cf1c4415eb5cc1269 src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java 348bdbfbb51800d31b67ea1b3f2632975e598306 src/main/java/org/apache/aurora/scheduler/updater/OneWayJobUpdater.java 1d0bbfef583708bfdab30802724273e4b21ecf8f src/main/java/org/apache/aurora/scheduler/updater/UpdateFactory.java 842256a16fe2775af865ed2ee91fd719b3476577 src/main/python/apache/aurora/client/api/__init__.py bbc8cf0314388b5b7cfe05562e8f9391fd722ac1 src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml e1179b449c05adc56206bfef271a9d440a77e042 src/main/thrift/org/apache/aurora/gen/api.thrift 2376a5e530b12fbbebb4cfc7555656ae07795518 src/test/java/org/apache/aurora/scheduler/storage/db/DBJobUpdateStoreTest.java 553ef4d88fafff894a11bc6fb6d5ab33d5f6646d src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 21e8a8ba5c47f549133ea339899bf0e5097ccbb9 src/test/java/org/apache/aurora/scheduler/updater/JobDiffTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/updater/JobUpdaterIT.java f7c7205d80edac714a12c2aff0cc7db27b1909b4 src/test/java/org/apache/aurora/scheduler/updater/OneWayJobUpdaterTest.java ad4040c4a58d3e6f075a60712144aa6968270a55 src/test/java/org/apache/aurora/scheduler/updater/UpdateFactoryImplTest.java 548d31580b5f2bae92900fddb80005f7d274a155 Diff: https://reviews.apache.org/r/25969/diff/ Testing --- ./gradlew build -Pq Thanks, Bill Farner
Re: Review Request 25963: Expose in progress and recently completed updates in UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/#review54371 --- Ship it! src/main/resources/org/apache/aurora/scheduler/http/ui/js/services.js https://reviews.apache.org/r/25963/#comment94513 You could race maxim to make this a constant. See api.thrift in https://reviews.apache.org/r/25812/ src/main/resources/org/apache/aurora/scheduler/http/ui/js/services.js https://reviews.apache.org/r/25963/#comment94514 You could race maxim to make this a thrift constant. See api.thrift in https://reviews.apache.org/r/25812/ - Bill Farner On Sept. 23, 2014, 11:46 p.m., David McLaughlin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/ --- (Updated Sept. 23, 2014, 11:46 p.m.) Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner. Bugs: AURORA-745 https://issues.apache.org/jira/browse/AURORA-745 Repository: aurora Description --- Very simple first iteration. Shows only the latest 20 updates. Diffs - src/main/java/org/apache/aurora/scheduler/http/JettyServerModule.java 392b4f7bc9d5352ed536ff83eafb4175e4860aa1 src/main/resources/org/apache/aurora/scheduler/http/ui/css/app.css befd5908e51dcac2a0fc56b12651af11273d25c0 src/main/resources/org/apache/aurora/scheduler/http/ui/index.html 6434d1bc3b9f769f6f565d601fd98f99743efe8c src/main/resources/org/apache/aurora/scheduler/http/ui/js/app.js 082d920348f12b214b8ca2d20c11e7b4825453ed src/main/resources/org/apache/aurora/scheduler/http/ui/js/controllers.js ded175a1e4b288b8978235c986db2549c22dcc7d src/main/resources/org/apache/aurora/scheduler/http/ui/js/services.js f65cb5e69c2d1712b5f9da782ee2c7b3bc22cee8 src/main/resources/org/apache/aurora/scheduler/http/ui/updateList.html PRE-CREATION Diff: https://reviews.apache.org/r/25963/diff/ Testing --- ./gradlew jsHint File Attachments updates page https://reviews.apache.org/media/uploaded/files/2014/09/23/33b2b9ff-3485-4bba-a8e8-2d4e6280e5ca__Screen_Shot_2014-09-23_at_4.15.06_PM.png no data updates https://reviews.apache.org/media/uploaded/files/2014/09/23/5fe9c0bf-ef31-41a3-95c4-7aeb9abcb3b0__Screen_Shot_2014-09-23_at_4.45.59_PM.png Thanks, David McLaughlin
Re: Review Request 25958: Enable beta updater in example cluster.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25958/#review54372 --- Ship it! In the future, please allow everyone on the 'People' line a chance to chime in. - Bill Farner On Sept. 23, 2014, 10:08 p.m., David McLaughlin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25958/ --- (Updated Sept. 23, 2014, 10:08 p.m.) Review request for Aurora, Maxim Khutornenko and Bill Farner. Repository: aurora Description --- enable beta updater in example cluster Diffs - examples/vagrant/upstart/aurora-scheduler.conf 7e2b3f0191cbc9f0b2ce916a1ad0b55281d718d6 Diff: https://reviews.apache.org/r/25958/diff/ Testing --- $ vagrant provision $ vagrant ssh $ aurora2 update start devcluster/www-data/devel/hello_world hello_world.aurora log(info): Starting update for: hello_world log(info): Starting new HTTP connection (1): 192.168.33.7 log(info): Starting new HTTP connection (1): 192.168.33.7 Scheduler-driven update of job devcluster/www-data/devel/hello_world has started. Thanks, David McLaughlin
Review Request 25972: Make thermos more cognizant of user deletions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25972/ --- Review request for Aurora, Joe Smith and Kevin Sweeney. Bugs: AURORA-175 https://issues.apache.org/jira/browse/AURORA-175 Repository: aurora Description --- This is changes 1 of 2 for AURORA-175. The second change adds an exit status contract between Aurora and Thermos. This change allows a process in the Thermos state machine to go directly from WAITING - FAILED if the user associated with that process has been deleted. It also persists a 'uid' field to the Thermos RunnerHeader so that we can check against process UIDs which in theory have higher fidelity than usernames. Diffs - src/main/python/apache/thermos/common/ckpt.py 7df179b135e896d52655667f0707850a4bc33068 src/main/python/apache/thermos/core/helper.py 82c68f3af424d3309db3816080465edd1ff1a87c src/main/python/apache/thermos/core/process.py 4889e636c2fa7325852684cecd87a2123714144d src/main/python/apache/thermos/core/runner.py 31f40713ec32c0626566899caec76f2a9985c7bf src/main/thrift/org/apache/thermos/thermos_internal.thrift 0cea1057230b367c9d55b773854453cee2f9fce0 src/test/python/apache/thermos/core/test_helper.py 53f1e467283b45598b3f1865951c42b13579a512 src/test/python/apache/thermos/core/test_process.py 223393d62d85dbcd3da3c2ca0d5558e8209aa1d1 Diff: https://reviews.apache.org/r/25972/diff/ Testing --- Thanks, Brian Wickman
Re: Review Request 25963: Expose in progress and recently completed updates in UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/#review54373 --- Ship it! Ship It! - Maxim Khutornenko On Sept. 23, 2014, 11:46 p.m., David McLaughlin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25963/ --- (Updated Sept. 23, 2014, 11:46 p.m.) Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner. Bugs: AURORA-745 https://issues.apache.org/jira/browse/AURORA-745 Repository: aurora Description --- Very simple first iteration. Shows only the latest 20 updates. Diffs - src/main/java/org/apache/aurora/scheduler/http/JettyServerModule.java 392b4f7bc9d5352ed536ff83eafb4175e4860aa1 src/main/resources/org/apache/aurora/scheduler/http/ui/css/app.css befd5908e51dcac2a0fc56b12651af11273d25c0 src/main/resources/org/apache/aurora/scheduler/http/ui/index.html 6434d1bc3b9f769f6f565d601fd98f99743efe8c src/main/resources/org/apache/aurora/scheduler/http/ui/js/app.js 082d920348f12b214b8ca2d20c11e7b4825453ed src/main/resources/org/apache/aurora/scheduler/http/ui/js/controllers.js ded175a1e4b288b8978235c986db2549c22dcc7d src/main/resources/org/apache/aurora/scheduler/http/ui/js/services.js f65cb5e69c2d1712b5f9da782ee2c7b3bc22cee8 src/main/resources/org/apache/aurora/scheduler/http/ui/updateList.html PRE-CREATION Diff: https://reviews.apache.org/r/25963/diff/ Testing --- ./gradlew jsHint File Attachments updates page https://reviews.apache.org/media/uploaded/files/2014/09/23/33b2b9ff-3485-4bba-a8e8-2d4e6280e5ca__Screen_Shot_2014-09-23_at_4.15.06_PM.png no data updates https://reviews.apache.org/media/uploaded/files/2014/09/23/5fe9c0bf-ef31-41a3-95c4-7aeb9abcb3b0__Screen_Shot_2014-09-23_at_4.45.59_PM.png Thanks, David McLaughlin
Review Request 25974: Prevent uncaught exceptions from killing the executor.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25974/ --- Review request for Aurora, Kevin Sweeney, Bill Farner, and Brian Wickman. Bugs: AURORA-728 https://issues.apache.org/jira/browse/AURORA-728 Repository: aurora Description --- Prevent uncaught exceptions from killing the executor when creating the status manager. Diffs - src/main/python/apache/aurora/executor/aurora_executor.py 79a24855b2a68271b7478395dfdadab8755c3af2 Diff: https://reviews.apache.org/r/25974/diff/ Testing --- ./pants src/test/python/apache/aurora/executor:executor-small Thanks, Zameer Manji
Re: Review Request 25974: Prevent uncaught exceptions from killing the executor.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25974/#review54378 --- I'm not sure a TASK_FAILED: KeyError: 'getpwuid() uid:0 not found' is the best user experience. Perhaps log.error the full traceback and do TASK_FAILED: Internal error or something like that. I still think this does not solve AURORA-728. Perhaps add a reconnect loop and if it's not connected within the initial health interval + (max consecutive failures * health check interval), send a StatusResult of TASK_FAILED directly from the Announcer. - Brian Wickman On Sept. 24, 2014, 1:20 a.m., Zameer Manji wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25974/ --- (Updated Sept. 24, 2014, 1:20 a.m.) Review request for Aurora, Kevin Sweeney, Bill Farner, and Brian Wickman. Bugs: AURORA-728 https://issues.apache.org/jira/browse/AURORA-728 Repository: aurora Description --- Prevent uncaught exceptions from killing the executor when creating the status manager. Diffs - src/main/python/apache/aurora/executor/aurora_executor.py 79a24855b2a68271b7478395dfdadab8755c3af2 Diff: https://reviews.apache.org/r/25974/diff/ Testing --- ./pants src/test/python/apache/aurora/executor:executor-small Thanks, Zameer Manji
