Re: Review Request 35498: Compute SLA stats for non-prod jobs
On June 25, 2015, 7:59 p.m., Kevin Sweeney wrote: src/main/java/org/apache/aurora/scheduler/sla/MetricCalculator.java, lines 138-139 https://reviews.apache.org/r/35498/diff/7/?file=991483#file991483line138 Add getters for these fields and access them below via the getters rather than direct field access. Stephan Erb wrote: I can submit an updated patch tonight. I've somewhat expected that some of you would point it out :-) I thought about adding these when writing the patch but then decided against them: Getters for final attributes on an inner class did not seem to offer any meaningful encapsulation or help during refactoring. Kevin Sweeney wrote: In this case the class in question uses getters and direct field access inconsistently, I'd also be okay with removing all getters on that class and replacing them with direct field access. I think getters are more readable though (it's a code review red flag when I see a new direct field access). Other committers feel free to chime in here. Stephan Erb wrote: Your consistency argument is very valid. I guess we can leave it as it is. Maxim Khutornenko wrote: IMO, using getters in cases like this feels redundant and just creates ground for inconsistent field/method reference. I'd drop getters as they provide zero value here. I've changed the code back to not using any getters where possible. Anything still missing, or can we get this merged? - Stephan --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35498/#review89394 --- On July 1, 2015, 7:09 p.m., Stephan Erb wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35498/ --- (Updated July 1, 2015, 7:09 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-1350 https://issues.apache.org/jira/browse/AURORA-1350 Repository: aurora Description --- Compute SLA stats for non-prod jobs Diffs - NEWS fd5b70823cf7a695831d34453ab101c7d6a626e8 docs/sla.md 14e9108fda91200bbf56384c96b9cd926689311f src/main/java/org/apache/aurora/scheduler/sla/MetricCalculator.java 82f36d5ca15df18bdc8ebbbd868d3394db38e603 src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java ff73ca6265bd0699791da5e5b6ed4aab9156d9e4 src/main/java/org/apache/aurora/scheduler/sla/SlaModule.java 64e986fb2e0f955dd4a9c7824eac9494728bf24e src/test/java/org/apache/aurora/scheduler/sla/MetricCalculatorTest.java cb98834e925793fc116814371548a30470830164 src/test/java/org/apache/aurora/scheduler/sla/SlaModuleTest.java 5ee123a03e3c8670e0c03b05c48a9f4c66f6af9d Diff: https://reviews.apache.org/r/35498/diff/ Testing --- `./gradlew -Pq build` and a manual verification in Vagrant. Thanks, Stephan Erb
Re: Review Request 34337: Add Docker Parameters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34337/ --- (Updated July 5, 2015, 10:39 p.m.) Review request for Aurora and Bill Farner. Changes --- added argument to disable arbitrary docker parameters Repository: aurora Description --- Support Arbitrary Docker Parameters in DockerContainer Diffs (updated) - api/src/main/thrift/org/apache/aurora/gen/api.thrift d740a90e7732f42b43a79f8cf0afe705c061539c docs/configuration-reference.md 7bfd63381f54b0fe5ef6a4f17b825049b19038db src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java b77b0ebbf303778e528b16ff3db1aa4e76f1 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java e934f570e4a728470408970485abe0809487d312 src/main/python/apache/aurora/config/schema/base.py ec9f983564516afe542ab277d987d4d391f87e45 src/main/python/apache/aurora/config/thrift.py 810febb637d168b07c4aea77984e1d1451a39af2 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 1b2a7948ebb946a2e12b0eded6acf4ce3c8e20f9 Diff: https://reviews.apache.org/r/34337/diff/ Testing --- Used Docker as the container of a Job. Included volumes and label parameters which are correctly picked up by mesos when starting the task. The docker container gets the specified label and bind mounts the volumes correctly. I've been running multiple PostgreSQL databases docker containers for several weeks deploying them as aurora jobs. Thanks, Mauricio Garavaglia
Re: Review Request 34337: Add Docker Parameters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34337/ --- (Updated July 5, 2015, 11:58 p.m.) Review request for Aurora and Bill Farner. Repository: aurora Description --- Support Arbitrary Docker Parameters in DockerContainer Diffs (updated) - api/src/main/thrift/org/apache/aurora/gen/api.thrift d740a90 docs/configuration-reference.md dafd306 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java be79e70 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java c0d165a src/main/python/apache/aurora/config/schema/base.py d1f1e4f src/main/python/apache/aurora/config/thrift.py 88dd1c7 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c0cadfb Diff: https://reviews.apache.org/r/34337/diff/ Testing --- Used Docker as the container of a Job. Included volumes and label parameters which are correctly picked up by mesos when starting the task. The docker container gets the specified label and bind mounts the volumes correctly. I've been running multiple PostgreSQL databases docker containers for several weeks deploying them as aurora jobs. Thanks, Mauricio Garavaglia
Re: Review Request 34337: Add Docker Parameters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34337/ --- (Updated July 5, 2015, 11:32 p.m.) Review request for Aurora and Bill Farner. Changes --- rebased. @ReviewBot retry Repository: aurora Description --- Support Arbitrary Docker Parameters in DockerContainer Diffs (updated) - NEWS a17f0e7c08fd30a0b2db6814a1c755111307228b api/src/main/thrift/org/apache/aurora/gen/BUILD fe3f83b6a7680985dce01efe2d54ccc4b0c2c482 api/src/main/thrift/org/apache/aurora/gen/api.thrift d740a90e7732f42b43a79f8cf0afe705c061539c api/src/main/thrift/org/apache/aurora/gen/internal_rpc.thrift a2c230fa9b5f648c4674042411cbe46fb8bb4faa config/legacy_untested_classes.txt d2f3ca50d32abf945f58b5fb67cd27c0b127822d debian/aurora-executor.install 8efb1308caf64a23bed4b580de4b86e7982539e8 debian/aurora-scheduler.default 47fea5b4e6dadab9704c6ea5141268a91803fdc2 debian/aurora-scheduler.init 2ed0159d36d4be08f2c355dc1c21f36192a819f4 debian/aurora-scheduler.upstart ed7fe0917147d9c36a0828cc701da902142f72ba debian/rules 6ba18cef7fbf0989507d630a1041cdf958742617 docs/configuration-reference.md 7bfd63381f54b0fe5ef6a4f17b825049b19038db docs/cron-jobs.md 2f36dc97daf65e39adb554cc74981f40a7d64220 docs/storage-config.md 971bc1673f8be800f0d081a44018fc17a5ed025a docs/test-resource-generation.md 335586d64757f1e6293a89f14c1c3d578321eac6 examples/scheduler/scheduler-local.sh 6253d505f9851aea613bc95c15c313b03b57af11 examples/vagrant/aurorabuild.sh 5eb171cf45ffee1287f3ac039ab8cf3db6991a97 examples/vagrant/upstart/aurora-scheduler-kerberos.conf 414539b1917b5d33c577f1539575934c7f7c8167 examples/vagrant/upstart/aurora-scheduler.conf f4b867cbbcdbcc792518c2f90807834e47dce253 rbt c082ab4e0f541ff0d8c94719b027902ad1aacb17 src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 4c63cc7af2fb5702fa649ce2cd4ee619139223e2 src/jmh/java/org/apache/aurora/benchmark/Tasks.java d3e1295db9b933b5ab60d6446dfea453c6051795 src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java 3413b0a480073f39bc01a60bc93b211660d9e278 src/main/java/org/apache/aurora/GuavaUtils.java 22104e6d005ae6706a06a21ffe4c730b14ecfe53 src/main/java/org/apache/aurora/GuiceUtils.java 5d0af1ebdccd0d0782f0fbc53d046a338c450167 src/main/java/org/apache/aurora/auth/UnsecureSessionContext.java 9e1f35a12205065f2f0cc783a49e14384bc4d50a src/main/java/org/apache/aurora/scheduler/SchedulerModule.java 6edec22aba135136c4ce4066b9535f23de077db7 src/main/java/org/apache/aurora/scheduler/TaskLauncher.java cd55a6ee7424873c3e615d95422c9ecab6442f46 src/main/java/org/apache/aurora/scheduler/TaskStatusHandler.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/UserTaskLauncher.java 6bfbf0c76399d569ac762a7b433a6c576bf0ee87 src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java c31446c447c3385a4763b8a516827988e46cc480 src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 8bcac6c4f93eaca7250d43ab81a72102b91bf836 src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java f2ef70ddc5a859811f0e6c2ade62e115639c1654 src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java f0dea48b3df69dedf620a55573731ed9451369c3 src/main/java/org/apache/aurora/scheduler/async/preemptor/BiCache.java f5a18338748da7c443b2fd2ec3a72adf75e7387f src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterStateImpl.java cd016af5621be76190ead81921096aae837b59c0 src/main/java/org/apache/aurora/scheduler/async/preemptor/PendingTaskProcessor.java c1114a411c32c25785adc93e594ee8e291025969 src/main/java/org/apache/aurora/scheduler/async/preemptor/Preemptor.java 41591b80ee6d078e1c2beb5d98c7a7584ce173e2 src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 7f166e2ec24f66a9ab0c35e3c65af45e461943f9 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java b77b0ebbf303778e528b16ff3db1aa4e76f1 src/main/java/org/apache/aurora/scheduler/configuration/Resources.java 673a6909ec1bccbc43b0a3cd913aa48b9b1d90c1 src/main/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJob.java df180a4f9dff75a04ae104462c219444ef605d9a src/main/java/org/apache/aurora/scheduler/cron/quartz/CronJobManagerImpl.java 770672c85c06aaf4356b3b3580428b0323edb9d3 src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 0063aea16ff39e95c8d32848fdb3eeec6ab1bee3 src/main/java/org/apache/aurora/scheduler/filter/AttributeAggregate.java bd74f892dde65e957f125a17979ee6c582b3043f src/main/java/org/apache/aurora/scheduler/filter/ConstraintMatcher.java ecba276f1882162d9578e0829b2139040876c7d0 src/main/java/org/apache/aurora/scheduler/http/Cron.java fd658e1ac5e73ef0c61c82685f6373c5f0eb6640
Re: Review Request 33778: Adds RPM specfile and support scripting
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33778/ --- (Updated July 6, 2015, 12:32 a.m.) Review request for Aurora, Jake Farrell, Kevin Sweeney, and Bill Farner. Bugs: AURORA-1116 https://issues.apache.org/jira/browse/AURORA-1116 Repository: aurora Description --- This review board adds support for building Red Hat-flavored packages for the Aurora scheduler alongside its client and Thermos-related binaries: aurora - Aurora Scheduler aurora-client - Aurora client and admin tool aurora-thermos - Thermos executor, runner, and observer aurora-debuginfo - Debugging symbols for Python/libmesos integration If you'd like to give it a whirl, the following Make targets will spin up new RPMs: make (nightly_srpm|nightly_rpm) - builds an RPM or SRPM with timestamped versioning, suitable for producing nightly updates make (release_srpm|release_rpm) - builds without timestamped versioning Let me know what you think, and thanks! Diffs (updated) - build-support/packaging/rpm/Makefile PRE-CREATION build-support/packaging/rpm/README.md PRE-CREATION build-support/packaging/rpm/aurora.init.sh PRE-CREATION build-support/packaging/rpm/aurora.logrotate PRE-CREATION build-support/packaging/rpm/aurora.service PRE-CREATION build-support/packaging/rpm/aurora.spec PRE-CREATION build-support/packaging/rpm/aurora.startup.sh PRE-CREATION build-support/packaging/rpm/aurora.sysconfig PRE-CREATION build-support/packaging/rpm/clusters.json PRE-CREATION build-support/packaging/rpm/thermos-observer.init.sh PRE-CREATION build-support/packaging/rpm/thermos-observer.logrotate PRE-CREATION build-support/packaging/rpm/thermos-observer.service PRE-CREATION build-support/packaging/rpm/thermos-observer.startup.sh PRE-CREATION build-support/packaging/rpm/thermos-observer.sysconfig PRE-CREATION Diff: https://reviews.apache.org/r/33778/diff/ Testing --- Successfully executed builds on EL 6/7 and F19/20, tested deployment on EL6 and F19 Thanks, Steve Salevan
Re: Review Request 33778: Adds RPM specfile and support scripting
On June 9, 2015, 8:18 p.m., Kevin Sweeney wrote: packaging/rpm/aurora.service, line 21 https://reviews.apache.org/r/33778/diff/2/?file=947994#file947994line21 The scheduler should not run as root Added support for the creation of an aurora user/group as well as logic to support running the scheduler under this user. On June 9, 2015, 8:18 p.m., Kevin Sweeney wrote: packaging/rpm/aurora.spec, line 32 https://reviews.apache.org/r/33778/diff/2/?file=947995#file947995line32 support for 1.7 will be dropped in 0.9.0, I'd prefer we not get new users stuck here and remove this branch. Dropped. On June 9, 2015, 8:18 p.m., Kevin Sweeney wrote: packaging/rpm/aurora.spec, lines 28-34 https://reviews.apache.org/r/33778/diff/5/?file=963972#file963972line28 Java 8 is required in 0.9.0 - remove this branching entirely. Dropped. On June 9, 2015, 8:18 p.m., Kevin Sweeney wrote: packaging/rpm/aurora.spec, lines 166-171 https://reviews.apache.org/r/33778/diff/5/?file=963972#file963972line166 Drop this entirely now that the project is on JDK8? Dropped. On June 9, 2015, 8:18 p.m., Kevin Sweeney wrote: packaging/rpm/aurora.startup.sh, line 21 https://reviews.apache.org/r/33778/diff/5/?file=963973#file963973line21 the RHS is redundant here if you change GLOG_V to GLOG_v in the sysconfig file. Fixed. On June 9, 2015, 8:18 p.m., Kevin Sweeney wrote: packaging/rpm/aurora.startup.sh, lines 22-23 https://reviews.apache.org/r/33778/diff/5/?file=963973#file963973line22 RHS is redundant - you can just do ``` export GLOG_v LIBPROCESS_PORT LIBPROCES_IP ``` Fixed. On June 9, 2015, 8:18 p.m., Kevin Sweeney wrote: packaging/rpm/aurora.sysconfig, line 28 https://reviews.apache.org/r/33778/diff/5/?file=963974#file963974line28 Is this needed? This is the default right? Alas, the Mesosphere RPM drops the necessary libmesos bits in /usr/lib instead of /usr/lib64, which is the default on an EL system, so this forces the JVM libloader to look into /usr/lib instead. - Steve --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33778/#review82436 --- On May 19, 2015, 4:19 p.m., Steve Salevan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33778/ --- (Updated May 19, 2015, 4:19 p.m.) Review request for Aurora, Jake Farrell, Kevin Sweeney, and Bill Farner. Bugs: AURORA-1116 https://issues.apache.org/jira/browse/AURORA-1116 Repository: aurora Description --- This review board adds support for building Red Hat-flavored packages for the Aurora scheduler alongside its client and Thermos-related binaries: aurora - Aurora Scheduler aurora-client - Aurora client and admin tool aurora-thermos - Thermos executor, runner, and observer aurora-debuginfo - Debugging symbols for Python/libmesos integration If you'd like to give it a whirl, the following Make targets will spin up new RPMs: make (nightly_srpm|nightly_rpm) - builds an RPM or SRPM with timestamped versioning, suitable for producing nightly updates make (release_srpm|release_rpm) - builds without timestamped versioning Let me know what you think, and thanks! Diffs - packaging/rpm/Makefile PRE-CREATION packaging/rpm/aurora.init.sh PRE-CREATION packaging/rpm/aurora.logrotate PRE-CREATION packaging/rpm/aurora.service PRE-CREATION packaging/rpm/aurora.spec PRE-CREATION packaging/rpm/aurora.startup.sh PRE-CREATION packaging/rpm/aurora.sysconfig PRE-CREATION packaging/rpm/clusters.json PRE-CREATION packaging/rpm/thermos-observer.init.sh PRE-CREATION packaging/rpm/thermos-observer.logrotate PRE-CREATION packaging/rpm/thermos-observer.service PRE-CREATION packaging/rpm/thermos-observer.startup.sh PRE-CREATION packaging/rpm/thermos-observer.sysconfig PRE-CREATION Diff: https://reviews.apache.org/r/33778/diff/ Testing --- Successfully executed builds on EL 6/7 and F19/20, tested deployment on EL6 and F19 Thanks, Steve Salevan
Re: Review Request 33778: Adds RPM specfile and support scripting
On June 23, 2015, 2:55 p.m., Jake Farrell wrote: Are you able to update this Steve? would love to get this in for 0.9.0 The latest patch incorporates the above changes; let me know what you think. - Steve --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33778/#review88987 --- On May 19, 2015, 4:19 p.m., Steve Salevan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33778/ --- (Updated May 19, 2015, 4:19 p.m.) Review request for Aurora, Jake Farrell, Kevin Sweeney, and Bill Farner. Bugs: AURORA-1116 https://issues.apache.org/jira/browse/AURORA-1116 Repository: aurora Description --- This review board adds support for building Red Hat-flavored packages for the Aurora scheduler alongside its client and Thermos-related binaries: aurora - Aurora Scheduler aurora-client - Aurora client and admin tool aurora-thermos - Thermos executor, runner, and observer aurora-debuginfo - Debugging symbols for Python/libmesos integration If you'd like to give it a whirl, the following Make targets will spin up new RPMs: make (nightly_srpm|nightly_rpm) - builds an RPM or SRPM with timestamped versioning, suitable for producing nightly updates make (release_srpm|release_rpm) - builds without timestamped versioning Let me know what you think, and thanks! Diffs - packaging/rpm/Makefile PRE-CREATION packaging/rpm/aurora.init.sh PRE-CREATION packaging/rpm/aurora.logrotate PRE-CREATION packaging/rpm/aurora.service PRE-CREATION packaging/rpm/aurora.spec PRE-CREATION packaging/rpm/aurora.startup.sh PRE-CREATION packaging/rpm/aurora.sysconfig PRE-CREATION packaging/rpm/clusters.json PRE-CREATION packaging/rpm/thermos-observer.init.sh PRE-CREATION packaging/rpm/thermos-observer.logrotate PRE-CREATION packaging/rpm/thermos-observer.service PRE-CREATION packaging/rpm/thermos-observer.startup.sh PRE-CREATION packaging/rpm/thermos-observer.sysconfig PRE-CREATION Diff: https://reviews.apache.org/r/33778/diff/ Testing --- Successfully executed builds on EL 6/7 and F19/20, tested deployment on EL6 and F19 Thanks, Steve Salevan
Re: Review Request 33778: Adds RPM specfile and support scripting
On June 9, 2015, 3:49 p.m., Jake Farrell wrote: Patch looks good. Would like to see this move from top level /packaging to build-support/packaging and have docs/packaging.md or build-support/packaging/README.md added with some details for anyone that may not be familiar with dependencies needed to create the packages. With these updates will happily commit this patch, nice work (ultimately would like to see a gradle task for deb/rpm and maybe docker containers used to create the packages, but that is independant of this patch.) Added documentation and adjusted toplevel directory. - Steve --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33778/#review87208 --- On May 19, 2015, 4:19 p.m., Steve Salevan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33778/ --- (Updated May 19, 2015, 4:19 p.m.) Review request for Aurora, Jake Farrell, Kevin Sweeney, and Bill Farner. Bugs: AURORA-1116 https://issues.apache.org/jira/browse/AURORA-1116 Repository: aurora Description --- This review board adds support for building Red Hat-flavored packages for the Aurora scheduler alongside its client and Thermos-related binaries: aurora - Aurora Scheduler aurora-client - Aurora client and admin tool aurora-thermos - Thermos executor, runner, and observer aurora-debuginfo - Debugging symbols for Python/libmesos integration If you'd like to give it a whirl, the following Make targets will spin up new RPMs: make (nightly_srpm|nightly_rpm) - builds an RPM or SRPM with timestamped versioning, suitable for producing nightly updates make (release_srpm|release_rpm) - builds without timestamped versioning Let me know what you think, and thanks! Diffs - packaging/rpm/Makefile PRE-CREATION packaging/rpm/aurora.init.sh PRE-CREATION packaging/rpm/aurora.logrotate PRE-CREATION packaging/rpm/aurora.service PRE-CREATION packaging/rpm/aurora.spec PRE-CREATION packaging/rpm/aurora.startup.sh PRE-CREATION packaging/rpm/aurora.sysconfig PRE-CREATION packaging/rpm/clusters.json PRE-CREATION packaging/rpm/thermos-observer.init.sh PRE-CREATION packaging/rpm/thermos-observer.logrotate PRE-CREATION packaging/rpm/thermos-observer.service PRE-CREATION packaging/rpm/thermos-observer.startup.sh PRE-CREATION packaging/rpm/thermos-observer.sysconfig PRE-CREATION Diff: https://reviews.apache.org/r/33778/diff/ Testing --- Successfully executed builds on EL 6/7 and F19/20, tested deployment on EL6 and F19 Thanks, Steve Salevan
