Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 2, 2015, 9:03 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs (updated) - api/src/main/thrift/org/apache/aurora/gen/api.thrift 6f6124a4c844a1abcf07401d80c3e50eb8b4 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java bacfbfeb237ecddf82f58679e05be012c5214e61 src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
On March 2, 2015, 6:56 p.m., Bill Farner wrote: src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 133 https://reviews.apache.org/r/31338/diff/8/?file=882131#file882131line133 I think you can drop this since it's == `Resources.NONE` (default). Done On March 2, 2015, 6:56 p.m., Bill Farner wrote: src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java, line 137 https://reviews.apache.org/r/31338/diff/8/?file=882134#file882134line137 remove empty line Done On March 2, 2015, 6:56 p.m., Bill Farner wrote: src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java, lines 216-222 https://reviews.apache.org/r/31338/diff/8/?file=882138#file882138line216 This can collapse to: ``` assertTrue(taskInfo.getExecutor().getContainer().getVolumesList().contains(expected)); ``` Done - Steve --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review74782 --- On March 2, 2015, 4:59 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 2, 2015, 4:59 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift 6f6124a4c844a1abcf07401d80c3e50eb8b4 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java bacfbfeb237ecddf82f58679e05be012c5214e61 src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review74821 --- This patch does not apply cleanly on master (4cca6a6), do you need to rebase? I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 2, 2015, 9:03 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 2, 2015, 9:03 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift 6f6124a4c844a1abcf07401d80c3e50eb8b4 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java bacfbfeb237ecddf82f58679e05be012c5214e61 src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Review Request 31652: Returning pending reason for all tasks in a group
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31652/ --- Review request for Aurora, Bill Farner and Zameer Manji. Repository: aurora Description --- Modifying `Vetoed` event to broadcast `TaskGroupKey` instead of task ID and storing veto reasons by TaskGroupKey in NearestFit. Depends on https://reviews.apache.org/r/31646/. Diffs - src/main/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilter.java edaf2f4f845544c13b2fb9bc77c34f6e6d96fb48 src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java c103472b9404df1c690b3a6019d64d42e15f2fed src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 53582c63ddee23e643bd4654cad2bef75dfba36d src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 13520eb5846022ed0b43b402096fe02565103aa9 src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java ab7817f929bbcc96a6046043ea17921a388fdb9f src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java ce5a62650cebab9a53743460f5a5119f62efec1c Diff: https://reviews.apache.org/r/31652/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Review Request 31659: Clean up end-to-end tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs - src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31653: docs: Add info on granting production quota.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31653/#review74864 --- Ship it! Master (4cca6a6) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 12:58 a.m., Ricardo Cervera-Navarro wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31653/ --- (Updated March 3, 2015, 12:58 a.m.) Review request for Aurora and Bill Farner. Repository: aurora Description --- docs: Add info on granting production quota. Diffs - docs/configuration-reference.md 0da4a9b4c46843f4f810f9c61fee738568619ec6 docs/configuration-tutorial.md 0825d085b32b67127709111ca5c163a89992c839 Diff: https://reviews.apache.org/r/31653/diff/ Testing --- Thanks, Ricardo Cervera-Navarro
Re: Review Request 31525: Improving NearestFit reporting accuracy.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/ --- (Updated March 2, 2015, 11:40 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Changes --- Updating ticket number Bugs: AURORA-1159 https://issues.apache.org/jira/browse/AURORA-1159 Repository: aurora Description --- Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with break early rule in SchedulingFilter, this ensures heavier vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance. Diffs - src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147 src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424 src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 Diff: https://reviews.apache.org/r/31525/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31652: Returning pending reason for all tasks in a group
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31652/#review74865
---
Master (4cca6a6) is red with this patch.
./build-support/jenkins/build.sh
^
symbol: class TaskGroupKey
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java:97:
error: cannot find symbol
new FunctionITaskConfig, TaskGroupKey() {
^
symbol: class TaskGroupKey
location: class NearestFit
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java:99:
error: cannot find symbol
public TaskGroupKey apply(ITaskConfig task) {
^
symbol: class TaskGroupKey
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java:100:
error: cannot find symbol
return TaskGroupKey.from(task);
^
symbol: variable TaskGroupKey
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java:115:
error: cannot find symbol
fitByTask.invalidate(TaskGroupKey.from(event.getTask().getAssignedTask().getTask()));
^
symbol: variable TaskGroupKey
location: class NearestFit
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilter.java:61:
error: cannot find symbol
eventSink.post(new Vetoed(TaskGroupKey.from(request.getTask()), vetoes));
^
symbol: variable TaskGroupKey
location: class NotifyingSchedulingFilter
18 errors
FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':compileJava'.
Compilation failed; see the compiler error output for details.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output.
BUILD FAILED
Total time: 1 mins 20.724 secs
I will refresh this build result if you post a review containing @ReviewBot
retry
- Aurora ReviewBot
On March 3, 2015, 12:58 a.m., Maxim Khutornenko wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31652/
---
(Updated March 3, 2015, 12:58 a.m.)
Review request for Aurora, Bill Farner and Zameer Manji.
Bugs: AURORA-911
https://issues.apache.org/jira/browse/AURORA-911
Repository: aurora
Description
---
Modifying `Vetoed` event to broadcast `TaskGroupKey` instead of task ID and
storing veto reasons by TaskGroupKey in NearestFit.
Depends on https://reviews.apache.org/r/31646/.
Diffs
-
src/main/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilter.java
edaf2f4f845544c13b2fb9bc77c34f6e6d96fb48
src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java
c103472b9404df1c690b3a6019d64d42e15f2fed
src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java
c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java
53582c63ddee23e643bd4654cad2bef75dfba36d
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java
13520eb5846022ed0b43b402096fe02565103aa9
src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java
ab7817f929bbcc96a6046043ea17921a388fdb9f
src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java
78a236c0f9074692b67ce18e6e03f18fe4529e02
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java
ce5a62650cebab9a53743460f5a5119f62efec1c
Diff: https://reviews.apache.org/r/31652/diff/
Testing
---
./gradlew -Pq build
Thanks,
Maxim Khutornenko
Re: Review Request 31101: Native Debian packaging for Aurora
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31101/#review74880 --- debian/aurora-executor.thermos.default https://reviews.apache.org/r/31101/#comment121698 needs license header debian/aurora-executor.thermos.init https://reviews.apache.org/r/31101/#comment121697 Needs license header debian/aurora-scheduler.default https://reviews.apache.org/r/31101/#comment121701 needs license header debian/aurora-scheduler.init https://reviews.apache.org/r/31101/#comment121700 needs license header debian/aurora-scheduler.postinst https://reviews.apache.org/r/31101/#comment121699 needs license header debian/embed-runner-in-executor.py https://reviews.apache.org/r/31101/#comment121702 if we keep this file then it needs license header added debian/rules https://reviews.apache.org/r/31101/#comment121703 needs license header Bunch of files missing license headers as notes, but overall looks good. Would be nice to not have to put this at the top level, as wfarner pointed out also, but then would have to deal with symlinking and juggling to make it work (we did this in thrift before moving to the top level for ease). Would be good to wrap creation into a task that executed dch and dpkg commands all in one call - Jake Farrell On Feb. 17, 2015, 12:20 a.m., Benjamin Staffin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31101/ --- (Updated Feb. 17, 2015, 12:20 a.m.) Review request for Aurora and Jake Farrell. Bugs: AURORA-951 https://issues.apache.org/jira/browse/AURORA-951 Repository: aurora Description --- Native Debian packaging for Aurora This uses Debian package dependencies to install gradle and thrift-compiler. Gradle 2.2.1 isn't in Debian or Ubuntu yet, but you can build your own package easily enough using https://github.com/benley/gradle-packaging Diffs - debian/aurora-doc.docs PRE-CREATION debian/aurora-doc.examples PRE-CREATION debian/aurora-executor.dirs PRE-CREATION debian/aurora-executor.install PRE-CREATION debian/aurora-executor.links PRE-CREATION debian/aurora-executor.thermos.default PRE-CREATION debian/aurora-executor.thermos.init PRE-CREATION debian/aurora-executor.thermos.upstart PRE-CREATION debian/aurora-scheduler.default PRE-CREATION debian/aurora-scheduler.init PRE-CREATION debian/aurora-scheduler.install PRE-CREATION debian/aurora-scheduler.links PRE-CREATION debian/aurora-scheduler.postinst PRE-CREATION debian/aurora-scheduler.upstart PRE-CREATION debian/aurora-tools.install PRE-CREATION debian/aurora-tools.links PRE-CREATION debian/changelog PRE-CREATION debian/clusters.json PRE-CREATION debian/compat PRE-CREATION debian/control PRE-CREATION debian/copyright PRE-CREATION debian/embed-runner-in-executor.py PRE-CREATION debian/pants.ini PRE-CREATION debian/rules PRE-CREATION debian/source/format PRE-CREATION Diff: https://reviews.apache.org/r/31101/diff/ Testing --- Built debs in a clean Ubuntu 14.04 environment using git-buildpackage and pbuilder. Have been using debs built from this branch on a testing cluster for a few months now. Thanks, Benjamin Staffin
Review Request 31646: Moving GroupKey to scheduler.base.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31646/ --- Review request for Aurora and Bill Farner. Bugs: AURORA-911 https://issues.apache.org/jira/browse/AURORA-911 Repository: aurora Description --- Simple IDE-driven refactoring in preparation for the AURORA-911 fix. Diffs - src/main/java/org/apache/aurora/scheduler/async/OfferManager.java 7d3713972f1df54da4c8cbaae0fca59a8a6f3d77 src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java e5067e18af7c477d2927d83eacec063f3dec575a src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java 8cd6c966c9aca2e869311787fb5d5caba7439d3e src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java d0fe3e133cbec2418f31160bf8ab8adaa45bb958 src/main/java/org/apache/aurora/scheduler/base/TaskGroupKey.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/async/OfferManagerImplTest.java 0cbc71d50612323aa4d8acc33e74b879c0a76aff src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 4ee13c8e5d46ba863f4d9871884c7d494d07758d src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 87bc531d2a72f21c36ddd0c1bd3b2367826cc422 Diff: https://reviews.apache.org/r/31646/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31646: Moving GroupKey to scheduler.base.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31646/#review74861 --- Ship it! Master (4cca6a6) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 12:07 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31646/ --- (Updated March 3, 2015, 12:07 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-911 https://issues.apache.org/jira/browse/AURORA-911 Repository: aurora Description --- Simple IDE-driven refactoring in preparation for the AURORA-911 fix. Diffs - src/main/java/org/apache/aurora/scheduler/async/OfferManager.java 7d3713972f1df54da4c8cbaae0fca59a8a6f3d77 src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java e5067e18af7c477d2927d83eacec063f3dec575a src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java 8cd6c966c9aca2e869311787fb5d5caba7439d3e src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java d0fe3e133cbec2418f31160bf8ab8adaa45bb958 src/main/java/org/apache/aurora/scheduler/base/TaskGroupKey.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/async/OfferManagerImplTest.java 0cbc71d50612323aa4d8acc33e74b879c0a76aff src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 4ee13c8e5d46ba863f4d9871884c7d494d07758d src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 87bc531d2a72f21c36ddd0c1bd3b2367826cc422 Diff: https://reviews.apache.org/r/31646/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Review Request 31653: docs: Add info on granting production quota.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31653/ --- Review request for Aurora and Bill Farner. Repository: aurora Description --- docs: Add info on granting production quota. Diffs - docs/configuration-reference.md 0da4a9b4c46843f4f810f9c61fee738568619ec6 docs/configuration-tutorial.md 0825d085b32b67127709111ca5c163a89992c839 Diff: https://reviews.apache.org/r/31653/diff/ Testing --- Thanks, Ricardo Cervera-Navarro
Re: Review Request 31659: Clean up end-to-end tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review74881 --- Master (4cca6a6) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.client.cli.cron . SUCCESS src.test.python.apache.aurora.client.cli.inspect . SUCCESS src.test.python.apache.aurora.client.cli.job . SUCCESS src.test.python.apache.aurora.client.cli.plugins . SUCCESS src.test.python.apache.aurora.client.cli.quota . SUCCESS src.test.python.apache.aurora.client.cli.sla . SUCCESS src.test.python.apache.aurora.client.cli.supdate . SUCCESS src.test.python.apache.aurora.client.cli.task . SUCCESS src.test.python.apache.aurora.client.cli.update . SUCCESS src.test.python.apache.aurora.client.cli.version . SUCCESS src.test.python.apache.aurora.client.config . SUCCESS src.test.python.apache.aurora.client.factory . SUCCESS src.test.python.apache.aurora.client.hooks.hooked_api . SUCCESS src.test.python.apache.aurora.client.hooks.non_hooked_api . SUCCESS src.test.python.apache.aurora.common.test_aurora_job_key . SUCCESS src.test.python.apache.aurora.common.test_cluster . SUCCESS src.test.python.apache.aurora.common.test_cluster_option . SUCCESS src.test.python.apache.aurora.common.test_clusters . SUCCESS src.test.python.apache.aurora.common.test_http_signaler . SUCCESS src.test.python.apache.aurora.common.test_pex_version . SUCCESS src.test.python.apache.aurora.common.test_shellify . SUCCESS src.test.python.apache.aurora.common.test_transport . SUCCESS src.test.python.apache.aurora.config.test_base . SUCCESS src.test.python.apache.aurora.config.test_constraint_parsing . SUCCESS src.test.python.apache.aurora.config.test_loader . SUCCESS src.test.python.apache.aurora.config.test_thrift . SUCCESS src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . FAILURE src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 3:23 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 3:23 a.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko.
Re: Review Request 31659: Clean up end-to-end tests.
On March 3, 2015, 3:38 a.m., Aurora ReviewBot wrote: Master (4cca6a6) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.client.cli.cron . SUCCESS src.test.python.apache.aurora.client.cli.inspect . SUCCESS src.test.python.apache.aurora.client.cli.job . SUCCESS src.test.python.apache.aurora.client.cli.plugins . SUCCESS src.test.python.apache.aurora.client.cli.quota . SUCCESS src.test.python.apache.aurora.client.cli.sla . SUCCESS src.test.python.apache.aurora.client.cli.supdate . SUCCESS src.test.python.apache.aurora.client.cli.task . SUCCESS src.test.python.apache.aurora.client.cli.update . SUCCESS src.test.python.apache.aurora.client.cli.version . SUCCESS src.test.python.apache.aurora.client.config . SUCCESS src.test.python.apache.aurora.client.factory . SUCCESS src.test.python.apache.aurora.client.hooks.hooked_api . SUCCESS src.test.python.apache.aurora.client.hooks.non_hooked_api . SUCCESS src.test.python.apache.aurora.common.test_aurora_job_key . SUCCESS src.test.python.apache.aurora.common.test_cluster . SUCCESS src.test.python.apache.aurora.common.test_cluster_option . SUCCESS src.test.python.apache.aurora.common.test_clusters . SUCCESS src.test.python.apache.aurora.common.test_http_signaler . SUCCESS src.test.python.apache.aurora.common.test_pex_version . SUCCESS src.test.python.apache.aurora.common.test_shellify . SUCCESS src.test.python.apache.aurora.common.test_transport . SUCCESS src.test.python.apache.aurora.config.test_base . SUCCESS src.test.python.apache.aurora.config.test_constraint_parsing . SUCCESS src.test.python.apache.aurora.config.test_loader . SUCCESS src.test.python.apache.aurora.config.test_thrift . SUCCESS src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . FAILURE src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry Known flaky test AURORA-1054. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review74881 --- On March 3, 2015, 3:23 a.m., Bill Farner wrote: --- This is an
Re: Review Request 31525: Improving NearestFit reporting accuracy.
On March 2, 2015, 7:40 p.m., Bill Farner wrote: Can you elaborate on how this change would have addressed this scenario described in AURORA-1148? Seems like the confusion of only seeing 'Insufficient RAM' is not resolved by this change alone. Correct, it's an incremental quality improvement rather than a comprehensive fix (which I am not even sure is possible given our current NearestFit approach). The idea is to NOT emit lower score (aka better fit) vetoes until the higher score vetoes are dismissed. Here is the most likely execution sequence in AURORA-1148 case: 1. non-decicated hosts are filtered out (veto reason: dedicated constraint mismatch) 2. dedicated hosts are evaluated wrt diversity constraints (veto reason: host limit constraint mismatch) 3. dedicated host resources are evaluated (veto reason: insufficient RAM/CPU/DISK) Given that every VetoType triggered short-circuits further veto analysis, users would *more* likely see a host limit constraint. What this change does not solve is that in case there are some offers satisfying diversity analysis they would pollute veto reason with Insufficient resource veto thus humpering investigation. Given our current 10 minute veto reason expiration and default offer hold time (5 minutes), users would likely see pending reason occasionally flipping from host limit mismatch to insufficient resource. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/#review74801 --- On Feb. 27, 2015, 9:05 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/ --- (Updated Feb. 27, 2015, 9:05 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-1148 https://issues.apache.org/jira/browse/AURORA-1148 Repository: aurora Description --- Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with break early rule in SchedulingFilter, this ensures heavier vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance. Diffs - src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147 src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424 src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 Diff: https://reviews.apache.org/r/31525/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review74752 --- @ReviewBot retry - Steve Niemitz On March 2, 2015, 4:59 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 2, 2015, 4:59 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift 6f6124a4c844a1abcf07401d80c3e50eb8b4 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java bacfbfeb237ecddf82f58679e05be012c5214e61 src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review74751 --- Master (4cca6a6) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.client.cli.sla . SUCCESS src.test.python.apache.aurora.client.cli.supdate . SUCCESS src.test.python.apache.aurora.client.cli.task . SUCCESS src.test.python.apache.aurora.client.cli.update . SUCCESS src.test.python.apache.aurora.client.cli.version . SUCCESS src.test.python.apache.aurora.client.config . SUCCESS src.test.python.apache.aurora.client.factory . SUCCESS src.test.python.apache.aurora.client.hooks.hooked_api . SUCCESS src.test.python.apache.aurora.client.hooks.non_hooked_api . SUCCESS src.test.python.apache.aurora.common.test_aurora_job_key . SUCCESS src.test.python.apache.aurora.common.test_cluster . SUCCESS src.test.python.apache.aurora.common.test_cluster_option . SUCCESS src.test.python.apache.aurora.common.test_clusters . SUCCESS src.test.python.apache.aurora.common.test_http_signaler . SUCCESS src.test.python.apache.aurora.common.test_pex_version . SUCCESS src.test.python.apache.aurora.common.test_shellify . SUCCESS src.test.python.apache.aurora.common.test_transport . SUCCESS src.test.python.apache.aurora.config.test_base . SUCCESS src.test.python.apache.aurora.config.test_constraint_parsing . SUCCESS src.test.python.apache.aurora.config.test_loader . SUCCESS src.test.python.apache.aurora.config.test_thrift . SUCCESS src.test.python.apache.aurora.executor.common.announcer . SUCCESS src.test.python.apache.aurora.executor.common.directory_sandbox . SUCCESS src.test.python.apache.aurora.executor.common.executor_detector . SUCCESS src.test.python.apache.aurora.executor.common.executor_timeout . SUCCESS src.test.python.apache.aurora.executor.common.health_checker . FAILURE src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . SUCCESS src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 2, 2015, 4:59 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 2, 2015, 4:59 p.m.) Review request for Aurora, Jay Buffington and Bill Farner.
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
On Feb. 27, 2015, 5:01 p.m., Bill Farner wrote: src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java, lines 215-227 https://reviews.apache.org/r/31338/diff/4/?file=876499#file876499line215 Can this instead be an exact comparison of ListVolume? Presumably we should fail if other volumes show up. ``` assertEquals( ImmutableList.of(expected), taskInfo.getExecutor().getContainer().getVolumesList()); ``` Steve Niemitz wrote: I think checking only this specifically is a more robust test. There are other things we may want to mount in by default in the future (for example, we already mount the thermos run directory), and testing for all of them here would make the test to broad. Bill Farner wrote: Hmm...i see your point, but i don't entirely buy it :-) For example, if the code behind this erroneously inserts two `Protos.Volume` entries for each global mount, the test case should trip. Steve Niemitz wrote: imo it's more likely that we'll be adding more features around volumes (default or otherwise) than introducing breaking changes to the code that adds the default volumes. Unit tests that are too fragile I think cause more harm than good. Alrighty, that works for me! - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review74507 --- On March 2, 2015, 4:59 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 2, 2015, 4:59 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift 6f6124a4c844a1abcf07401d80c3e50eb8b4 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java bacfbfeb237ecddf82f58679e05be012c5214e61 src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31525: Improving NearestFit reporting accuracy.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/#review74793 --- src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java https://reviews.apache.org/r/31525/#comment121550 I don't think the Math.pow approach is necessary - why not make score an EnumMapVetoType, Integer (or just an EnumSetVetoType). A veto increments the counter in the bucket for its type (or sets the bit for it). Still very efficient and more type-safe. - Kevin Sweeney On Feb. 27, 2015, 1:05 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/ --- (Updated Feb. 27, 2015, 1:05 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-1148 https://issues.apache.org/jira/browse/AURORA-1148 Repository: aurora Description --- Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with break early rule in SchedulingFilter, this ensures heavier vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance. Diffs - src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147 src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424 src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 Diff: https://reviews.apache.org/r/31525/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31525: Improving NearestFit reporting accuracy.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/#review74801 --- Can you elaborate on how this change would have addressed this scenario described in AURORA-1148? Seems like the confusion of only seeing 'Insufficient RAM' is not resolved by this change alone. - Bill Farner On Feb. 27, 2015, 9:05 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/ --- (Updated Feb. 27, 2015, 9:05 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-1148 https://issues.apache.org/jira/browse/AURORA-1148 Repository: aurora Description --- Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with break early rule in SchedulingFilter, this ensures heavier vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance. Diffs - src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147 src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424 src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 Diff: https://reviews.apache.org/r/31525/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review74782 --- Ship it! LGTM mod nits. src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java https://reviews.apache.org/r/31338/#comment121541 I think you can drop this since it's == `Resources.NONE` (default). src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java https://reviews.apache.org/r/31338/#comment121540 remove empty line src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java https://reviews.apache.org/r/31338/#comment121537 This can collapse to: ``` assertTrue(taskInfo.getExecutor().getContainer().getVolumesList().contains(expected)); ``` - Bill Farner On March 2, 2015, 4:59 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 2, 2015, 4:59 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift 6f6124a4c844a1abcf07401d80c3e50eb8b4 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java bacfbfeb237ecddf82f58679e05be012c5214e61 src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31525: Improving NearestFit reporting accuracy.
On March 2, 2015, 7:14 p.m., Kevin Sweeney wrote: src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, line 64 https://reviews.apache.org/r/31525/diff/2/?file=880724#file880724line64 I don't think the Math.pow approach is necessary - why not make score an EnumMapVetoType, Integer (or just an EnumSetVetoType). A veto increments the counter in the bucket for its type (or sets the bit for it). Still very efficient and more type-safe. Not sure I understand the proposal. The score buckets, as currently defined, flatten out the VetoType hierarchy via non-intersecting int ranges. This abstracts out the internal VetoType relationship and simplifies the consuming side (NearestFit). The sum of all previous level veto scores should never exceed the next level score. I don't see how we can get by with EnumSetVetoType here without making NearestFit aware of VetoType hierarchy. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/#review74793 --- On Feb. 27, 2015, 9:05 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/ --- (Updated Feb. 27, 2015, 9:05 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-1148 https://issues.apache.org/jira/browse/AURORA-1148 Repository: aurora Description --- Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with break early rule in SchedulingFilter, this ensures heavier vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance. Diffs - src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147 src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424 src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 Diff: https://reviews.apache.org/r/31525/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
