Re: Review Request 37801: Added support for {{mesos.hostname}} in aur files
On Aug. 26, 2015, 7:55 p.m., Zameer Manji wrote:
The ticket referenced also talks about {{mesos.ip}}, will you also add
support for that?
Steve Niemitz wrote:
It's not exposed anywhere by mesos AFAIK. If someone can correct me I
can add it in another review.
I have to agree, looks like Mesos does not export it [1]. I think this review
is therefore ready to go. I will update my original ticket to drop the need for
ip. We can implement that on our own within the container once the hostname has
been exposed.
[1]
https://github.com/apache/mesos/blob/8a01dd8d2a6517d2fcf2983a436c5ce37e3666bf/include/mesos/mesos.proto#L848.
- Stephan
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37801/#review96565
---
On Aug. 28, 2015, 12:48 a.m., Steve Niemitz wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37801/
---
(Updated Aug. 28, 2015, 12:48 a.m.)
Review request for Aurora and Zameer Manji.
Bugs: AURORA-1261
https://issues.apache.org/jira/browse/AURORA-1261
Repository: aurora
Description
---
Added support for {{mesos.hostname}} in aur files
Diffs
-
src/main/python/apache/aurora/config/schema/base.py
d1f1e4f8082553f3a773e70a7f5cd0c48181ae8c
src/main/python/apache/aurora/config/thrift.py
88dd1c7cf2e5a9bfbe827bc342d49f7a18bc8558
src/main/python/apache/aurora/executor/common/task_info.py
082947548d8d8e0eaf2ee4807636f50237e0ce22
src/test/python/apache/aurora/config/test_base.py
5a58dd0f072889a850153841f972dcca4d3c913b
src/test/python/apache/aurora/config/test_thrift.py
f48ac88c076c12541626b497ca4fc47e5941dd31
Diff: https://reviews.apache.org/r/37801/diff/
Testing
---
Thanks,
Steve Niemitz
Re: Review Request 37789: Refer to shared task_configs table for job updates.
On Aug. 26, 2015, 11:18 a.m., Maxim Khutornenko wrote: Any noticable impact on JobDetailsBenchmark after this change? Thanks for the nudge - turns out this patch broke that test due to schema requirements. Fixed, and will put results in the testing field. On Aug. 26, 2015, 11:18 a.m., Maxim Khutornenko wrote: src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql, line 194 https://reviews.apache.org/r/37789/diff/1/?file=1053036#file1053036line194 kill newline Done. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37789/#review96563 --- On Aug. 25, 2015, 11:41 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37789/ --- (Updated Aug. 25, 2015, 11:41 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-647 https://issues.apache.org/jira/browse/AURORA-647 Repository: aurora Description --- Unfortunately the reference forced me to move the location of tables in schema.sql, making the delta look much bigger there. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStore.java 7728684560d3111edaa2e29d9738f71bc1d7a9ff src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java 02ea355539d6f56c1c861ac83293b649285f43f4 src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TaskConfigTypeHandler.java 2f58357dd63c46eb69de6a6daedcf027712d7b2e src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 9afc3f3fcdcc1b60af2207888993411a55984c8e src/main/java/org/apache/aurora/scheduler/storage/db/views/DbInstanceTaskConfig.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdate.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateDetails.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateInstructions.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbStoredJobUpdateDetails.java PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 6ffb54f5beef332097fd5c2390b56873a82f0aa9 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql d971aa19982e6095fbf514714df670cabd523d9d src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java 3e78c097a7a9252ded8a4a7fc6609ecf5d61c5b5 Diff: https://reviews.apache.org/r/37789/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37789: Refer to shared task_configs table for job updates.
On Aug. 26, 2015, 10:38 a.m., Kevin Sweeney wrote: src/main/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStore.java, line 162 https://reviews.apache.org/r/37789/diff/1/?file=1053026#file1053026line162 Inline this method reference. Done. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37789/#review96558 --- On Aug. 25, 2015, 11:41 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37789/ --- (Updated Aug. 25, 2015, 11:41 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-647 https://issues.apache.org/jira/browse/AURORA-647 Repository: aurora Description --- Unfortunately the reference forced me to move the location of tables in schema.sql, making the delta look much bigger there. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStore.java 7728684560d3111edaa2e29d9738f71bc1d7a9ff src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java 02ea355539d6f56c1c861ac83293b649285f43f4 src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TaskConfigTypeHandler.java 2f58357dd63c46eb69de6a6daedcf027712d7b2e src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 9afc3f3fcdcc1b60af2207888993411a55984c8e src/main/java/org/apache/aurora/scheduler/storage/db/views/DbInstanceTaskConfig.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdate.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateDetails.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateInstructions.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbStoredJobUpdateDetails.java PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 6ffb54f5beef332097fd5c2390b56873a82f0aa9 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql d971aa19982e6095fbf514714df670cabd523d9d src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java 3e78c097a7a9252ded8a4a7fc6609ecf5d61c5b5 Diff: https://reviews.apache.org/r/37789/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37789: Refer to shared task_configs table for job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37789/#review96995 --- Ship it! Master (356eeac) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On Aug. 29, 2015, 3:45 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37789/ --- (Updated Aug. 29, 2015, 3:45 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Bugs: AURORA-647 https://issues.apache.org/jira/browse/AURORA-647 Repository: aurora Description --- Unfortunately the reference forced me to move the location of tables in schema.sql, making the delta look much bigger there. Diffs - src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 48bee5068db5aeacf181c76c4d3add71d045c8a9 src/main/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStore.java fd46c3f0718c7c60b7b80b903545490cd7727204 src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java 02ea355539d6f56c1c861ac83293b649285f43f4 src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TaskConfigTypeHandler.java 2f58357dd63c46eb69de6a6daedcf027712d7b2e src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 9afc3f3fcdcc1b60af2207888993411a55984c8e src/main/java/org/apache/aurora/scheduler/storage/db/views/DbInstanceTaskConfig.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdate.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateDetails.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateInstructions.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/views/DbStoredJobUpdateDetails.java PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 6ffb54f5beef332097fd5c2390b56873a82f0aa9 src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql d971aa19982e6095fbf514714df670cabd523d9d src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java 3e78c097a7a9252ded8a4a7fc6609ecf5d61c5b5 Diff: https://reviews.apache.org/r/37789/diff/ Testing --- Relevant benchmark impact is minimal: Before this change: ``` Benchmark (instances) Mode Cnt ScoreError Units UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 347.863 ± 77.177 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 77.135 ± 13.291 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 37.630 ± 5.200 ops/s ``` After this change: ``` Benchmark (instances) Mode Cnt Score Error Units UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt5 308.792 ± 159.968 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt5 71.926 ± 35.703 ops/s UpdateStoreBenchmarks.JobDetailsBenchmark.run1 thrpt5 41.472 ± 10.632 ops/s ``` Thanks, Bill Farner
Review Request 37924: Deb: move clusters.json from aurora-executor to aurora-tools package.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37924/ --- Review request for Aurora and Kevin Sweeney. Repository: aurora-packaging Description --- Deb: move clusters.json from aurora-executor to aurora-tools package. Diffs - specs/debian/aurora-executor.install 5d0d1f75b56ff0c8064fe02d516a6e257168 specs/debian/aurora-tools.install ac8d032d3c6f86ba8e9f73ea7bc712ebde7531f8 Diff: https://reviews.apache.org/r/37924/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 37825: Adding minimal implementation of the external tier config.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37825/ --- (Updated Aug. 29, 2015, 6:34 p.m.) Review request for Aurora, Bill Farner and Zameer Manji. Changes --- Bill's comments. Bugs: AURORA-1437 https://issues.apache.org/jira/browse/AURORA-1437 Repository: aurora Description --- The external config file is optional for now as tiers are not fully defined yet. Diffs (updated) - examples/vagrant/upstart/aurora-scheduler-kerberos.conf 744b4a35c61e749734e222b3d4cbd296927665aa examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff src/jmh/java/org/apache/aurora/benchmark/Offers.java 26bad1cce987ef7f46368a5936cf056aeb2f63b1 src/main/java/org/apache/aurora/scheduler/SchedulerModule.java d4bc6b9bb10c982fb6a2458d7fdd12bddbe98eca src/main/java/org/apache/aurora/scheduler/TierInfo.java 61bf30a7f11d2d4b9e49c58a6ed9ecd779d7e5ba src/main/java/org/apache/aurora/scheduler/TierManager.java 652afece17a7eb09b0ca68066707b1b8fbf024f0 src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java d7f3c60d383cf10afb0c0fcf4fe29972b183458c src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 399f58de6196b97abd359ecef8131b63480d591a src/test/java/org/apache/aurora/scheduler/ResourcesTest.java eb1baedcb13c2f169d819d137f22cb8b88db149c src/test/java/org/apache/aurora/scheduler/SchedulerModuleTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/TierManagerTest.java eb0207114e1b93968cb65832f154a9cd3bc3232e src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java adcc7a823ecf30442016ecbdd655622d6aeba65e src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 819b51e4c314749dc48db25693503af7d1ed0c54 src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java d8a524d98771ee68d7b4d423fb34e28101a04d27 src/test/resources/org/apache/aurora/scheduler/tiers-example.json PRE-CREATION Diff: https://reviews.apache.org/r/37825/diff/ Testing --- ./gradlew -Pq build ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh Thanks, Maxim Khutornenko
Review Request 37926: Adding oversubscription e2e tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37926/ --- Review request for Aurora, Bill Farner and Zameer Manji. Repository: aurora Description --- Also, had to backout earlier tier validation on the client until the feature exists beta. May not apply cleanly, diffed against 37825. Diffs - examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff examples/vagrant/upstart/mesos-slave.conf 2b6a60673fc0a7ea3b73471701cd5d3efd6ce639 src/main/java/org/apache/aurora/scheduler/http/Offers.java 4329ce10189df41a7a45cd6cc369333d7d5f65b0 src/main/python/apache/aurora/client/config.py f5ac084ae5b448221363add9476334742b217b92 src/test/python/apache/aurora/client/test_config.py 37459f5218e76fac831601806a0331927c0ee2c0 src/test/sh/org/apache/aurora/e2e/http/http_example_revocable.aurora PRE-CREATION src/test/sh/org/apache/aurora/e2e/http/http_example_revocable_updated.aurora PRE-CREATION src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 018efbe43de0c829799bc2f789958c55b1cb61a4 Diff: https://reviews.apache.org/r/37926/diff/ Testing --- Thanks, Maxim Khutornenko
Re: Review Request 37926: Adding oversubscription e2e tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37926/ --- (Updated Aug. 29, 2015, 9:04 p.m.) Review request for Aurora, Bill Farner and Zameer Manji. Repository: aurora Description (updated) --- Also, had to backout client tier validation added earlier until the feature exits beta. May not apply cleanly, diffed against 37825. Diffs - examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff examples/vagrant/upstart/mesos-slave.conf 2b6a60673fc0a7ea3b73471701cd5d3efd6ce639 src/main/java/org/apache/aurora/scheduler/http/Offers.java 4329ce10189df41a7a45cd6cc369333d7d5f65b0 src/main/python/apache/aurora/client/config.py f5ac084ae5b448221363add9476334742b217b92 src/test/python/apache/aurora/client/test_config.py 37459f5218e76fac831601806a0331927c0ee2c0 src/test/sh/org/apache/aurora/e2e/http/http_example_revocable.aurora PRE-CREATION src/test/sh/org/apache/aurora/e2e/http/http_example_revocable_updated.aurora PRE-CREATION src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 018efbe43de0c829799bc2f789958c55b1cb61a4 Diff: https://reviews.apache.org/r/37926/diff/ Testing --- Thanks, Maxim Khutornenko
