Re: Review Request 23455: Add a lightweight version of command hooks for clientv1 commands.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23455/#review48362 --- src/main/python/apache/aurora/client/commands/core.py https://reviews.apache.org/r/23455/#comment84911 Yeah, I know - but I think that it's clearer. Probably just me, but I find that skimming code, I look for something in the body of a function, and I misread subsequent code as being nested functions if the explicit pass is omitted. - Mark Chu-Carroll On July 14, 2014, 1:50 p.m., Mark Chu-Carroll wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23455/ --- (Updated July 14, 2014, 1:50 p.m.) Review request for Aurora, Joe Smith and Kevin Sweeney. Bugs: aurora-581 https://issues.apache.org/jira/browse/aurora-581 Repository: aurora Description --- Add a lightweight version of command hooks for clientv1 commands. Diffs - src/main/python/apache/aurora/client/commands/core.py 0939f0570572375ba9b4d054ddb9849995c9048d src/test/python/apache/aurora/client/commands/test_kill.py 94bbe1bfff1c8d333837aa6bb14700714f46288b Diff: https://reviews.apache.org/r/23455/diff/ Testing --- Added new unit tests, and ran full unit test suite. Thanks, Mark Chu-Carroll
Re: Review Request 23244: Fixing return value in _check_sla.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/ --- (Updated July 22, 2014, 4:41 p.m.) Review request for Aurora, Joe Smith and Mark Chu-Carroll. Changes --- Added more logging. Bugs: AURORA-571 https://issues.apache.org/jira/browse/AURORA-571 Repository: aurora Description --- Fixing return value in _check_sla. Also, adding logging in drainHosts RPC to help troubleshooting occasional LOST tasks when draining. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/state/MaintenanceController.java bf48db97cf882105c0b2e6b70f9a4612ce60dbe8 src/main/python/apache/aurora/admin/host_maintenance.py b5b1375b1bbaed833acf2d17ec432b3d99145fab src/test/python/apache/aurora/client/commands/test_maintenance.py c4677fc2cf4cfa792823cd0660c13371ec28301f Diff: https://reviews.apache.org/r/23244/diff/ Testing --- gradle -Pq build ./pants src/test/python:all Thanks, Maxim Khutornenko
Re: Review Request 23817: Do not rely upon pytest capsys since command line parameters alter outcome.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23817/#review48381 --- Ship it! Ship It! - Kevin Sweeney On July 22, 2014, 9:27 a.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23817/ --- (Updated July 22, 2014, 9:27 a.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-595 https://issues.apache.org/jira/browse/AURORA-595 Repository: aurora Description --- Do not rely upon pytest capsys since command line parameters alter outcome. Diffs - src/test/python/apache/aurora/common/test_cluster_option.py 2213d8cda42566dd015217de06be8d0572edb649 Diff: https://reviews.apache.org/r/23817/diff/ Testing --- ./pants src/test/python/apache/aurora/common:test_cluster_option -s Thanks, Brian Wickman
Re: Review Request 23823: Upgrade jersey version.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23823/#review48413 --- Ship it! Ship It! - Maxim Khutornenko On July 22, 2014, 7:42 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23823/ --- (Updated July 22, 2014, 7:42 p.m.) Review request for Aurora and Maxim Khutornenko. Repository: aurora Description --- This reimplements the trivial class com.twitter.common.net.http.filters.AbstractHttpFilter, which was the only source of a transitive jersey dependency. Diffs - build.gradle 32a31e66d309789f074d118fddeede567a56a37e config/pmd/design.xml fb86e0a61c4f2c75df8de8f3a367bba2eb0a38f3 src/main/java/org/apache/aurora/scheduler/http/AbstractFilter.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/http/CorsFilter.java 5b2e7354c6ed5b1f3f0e740c4e083584b63d89ae src/main/java/org/apache/aurora/scheduler/http/HttpStatsFilter.java f7f337f1a843def17957320ef3b662a440ca92a8 src/main/java/org/apache/aurora/scheduler/http/LeaderRedirectFilter.java 86939aa945f39be9096d2d05ccec3bf60973cb15 Diff: https://reviews.apache.org/r/23823/diff/ Testing --- ./gradlew clean build -Pq Thanks, Bill Farner
Re: Review Request 23254: Refactoring SchedulerCore (killTasks)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23254/#review48414 --- Ship it! Ship It! - Kevin Sweeney On July 21, 2014, 4:23 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23254/ --- (Updated July 21, 2014, 4:23 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-94 https://issues.apache.org/jira/browse/AURORA-94 Repository: aurora Description --- Splitting the earlier RB for easier reviewing. Addressing killTasks refactoring. Also moving some useful tests out of BaseSchedulerCoreImplTest. Diffs - src/main/java/org/apache/aurora/scheduler/http/ServletModule.java 27599f75603542069084631baf9195b8ad75e902 src/main/java/org/apache/aurora/scheduler/state/SchedulerCore.java 628464de0032db4eb5884e3b74346b2390408993 src/main/java/org/apache/aurora/scheduler/state/SchedulerCoreImpl.java b6d06f39ac39570eac4495d97d3adaabe8357bf7 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 95ee7b784889d66cae9b714bd88c27a136a54b3a src/main/java/org/apache/aurora/scheduler/thrift/Util.java 4ebb608d28b667bf47573524aa8878ac3b21ecfe src/main/java/org/apache/aurora/scheduler/thrift/aop/LoggingInterceptor.java 8692051fae08a59e4b70b330eb6723f72ff04d43 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java 2298971ffac45a284f9130e2122aeea8b39dc852 src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java d2bd65eef5a8ceea92dd62044e5e2c621c4a4f3e src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java 94eb0a22037187625636b24707d4ac6741b55f22 src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java b28761d5bbfdd2fea4714c79b47be5f5eb3a5280 Diff: https://reviews.apache.org/r/23254/diff/ Testing --- gradle -Pq clean build Thanks, Maxim Khutornenko
Re: Review Request 23244: Fixing return value in _check_sla.
On July 22, 2014, 7:59 p.m., Mark Chu-Carroll wrote: src/main/python/apache/aurora/admin/host_maintenance.py, line 131 https://reviews.apache.org/r/23244/diff/2/?file=639321#file639321line131 What is this initialization doing? I don't see what it changes - it's reassigned just below, in code which executes unconditionally. Ah, that's the actual bug I fixed. In case the inner IF block does not return, the control falls through to the method return statement, which blows with 'unsafe_hostnames' is not defined error. - Maxim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/#review48415 --- On July 22, 2014, 4:41 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/ --- (Updated July 22, 2014, 4:41 p.m.) Review request for Aurora, Joe Smith and Mark Chu-Carroll. Bugs: AURORA-571 https://issues.apache.org/jira/browse/AURORA-571 Repository: aurora Description --- Fixing return value in _check_sla. Also, adding logging in drainHosts RPC to help troubleshooting occasional LOST tasks when draining. Diffs - src/main/java/org/apache/aurora/scheduler/state/MaintenanceController.java bf48db97cf882105c0b2e6b70f9a4612ce60dbe8 src/main/python/apache/aurora/admin/host_maintenance.py b5b1375b1bbaed833acf2d17ec432b3d99145fab src/test/python/apache/aurora/client/commands/test_maintenance.py c4677fc2cf4cfa792823cd0660c13371ec28301f Diff: https://reviews.apache.org/r/23244/diff/ Testing --- gradle -Pq build ./pants src/test/python:all Thanks, Maxim Khutornenko
Re: Review Request 23244: Fixing return value in _check_sla.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/#review48418 --- src/main/python/apache/aurora/admin/host_maintenance.py https://reviews.apache.org/r/23244/#comment84984 Oh, I see. So why not just do if host_groups and len(host_groups) 1? - Mark Chu-Carroll On July 22, 2014, 12:41 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/ --- (Updated July 22, 2014, 12:41 p.m.) Review request for Aurora, Joe Smith and Mark Chu-Carroll. Bugs: AURORA-571 https://issues.apache.org/jira/browse/AURORA-571 Repository: aurora Description --- Fixing return value in _check_sla. Also, adding logging in drainHosts RPC to help troubleshooting occasional LOST tasks when draining. Diffs - src/main/java/org/apache/aurora/scheduler/state/MaintenanceController.java bf48db97cf882105c0b2e6b70f9a4612ce60dbe8 src/main/python/apache/aurora/admin/host_maintenance.py b5b1375b1bbaed833acf2d17ec432b3d99145fab src/test/python/apache/aurora/client/commands/test_maintenance.py c4677fc2cf4cfa792823cd0660c13371ec28301f Diff: https://reviews.apache.org/r/23244/diff/ Testing --- gradle -Pq build ./pants src/test/python:all Thanks, Maxim Khutornenko
Re: Review Request 23244: Fixing return value in _check_sla.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/ --- (Updated July 22, 2014, 8:40 p.m.) Review request for Aurora, Joe Smith and Mark Chu-Carroll. Changes --- CR comments. Bugs: AURORA-571 https://issues.apache.org/jira/browse/AURORA-571 Repository: aurora Description --- Fixing return value in _check_sla. Also, adding logging in drainHosts RPC to help troubleshooting occasional LOST tasks when draining. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/state/MaintenanceController.java bf48db97cf882105c0b2e6b70f9a4612ce60dbe8 src/main/python/apache/aurora/admin/host_maintenance.py b5b1375b1bbaed833acf2d17ec432b3d99145fab src/test/python/apache/aurora/client/commands/test_maintenance.py c4677fc2cf4cfa792823cd0660c13371ec28301f Diff: https://reviews.apache.org/r/23244/diff/ Testing --- gradle -Pq build ./pants src/test/python:all Thanks, Maxim Khutornenko
Re: Review Request 23244: Fixing return value in _check_sla.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/#review48427 --- Ship it! Ship It! src/main/python/apache/aurora/admin/host_maintenance.py https://reviews.apache.org/r/23244/#comment85019 Doh - I misread the indents. I hate python. - Mark Chu-Carroll On July 22, 2014, 4:40 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23244/ --- (Updated July 22, 2014, 4:40 p.m.) Review request for Aurora, Joe Smith and Mark Chu-Carroll. Bugs: AURORA-571 https://issues.apache.org/jira/browse/AURORA-571 Repository: aurora Description --- Fixing return value in _check_sla. Also, adding logging in drainHosts RPC to help troubleshooting occasional LOST tasks when draining. Diffs - src/main/java/org/apache/aurora/scheduler/state/MaintenanceController.java bf48db97cf882105c0b2e6b70f9a4612ce60dbe8 src/main/python/apache/aurora/admin/host_maintenance.py b5b1375b1bbaed833acf2d17ec432b3d99145fab src/test/python/apache/aurora/client/commands/test_maintenance.py c4677fc2cf4cfa792823cd0660c13371ec28301f Diff: https://reviews.apache.org/r/23244/diff/ Testing --- gradle -Pq build ./pants src/test/python:all Thanks, Maxim Khutornenko
Review Request 23827: Filtering out unknown hosts when starting maintenance.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23827/ --- Review request for Aurora and Mark Chu-Carroll. Bugs: AURORA-443 https://issues.apache.org/jira/browse/AURORA-443 Repository: aurora Description --- Filtering out unknown hosts when starting maintenance. Diffs - src/main/python/apache/aurora/admin/host_maintenance.py af26f7c831366307d47cf1d9e1d58f54abc30368 src/test/python/apache/aurora/admin/test_host_maintenance.py 34578ae82992d4b4f3c507417dd0258e7b0337bf src/test/python/apache/aurora/client/commands/test_maintenance.py 07dd3332eb5609fda875983653e6bc8e747f4ff2 Diff: https://reviews.apache.org/r/23827/diff/ Testing --- ./pants src/test/python:all Thanks, Maxim Khutornenko
Re: Review Request 23741: Add a human-friendly HTTP/JSON API with auto-generated documentation.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/#review48442
---
Overall I think this is great. For my own curiosity - any reason why you've
used gson over the simple JSON protocol that comes with Thrift?
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
https://reviews.apache.org/r/23741/#comment85048
We could probably remove this message.. I'm not sure it will have the
desired effect. On the subject of testing though, is it worth hitting this API
in the end to end tests?
- David McLaughlin
On July 21, 2014, 4:59 p.m., Bill Farner wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/
---
(Updated July 21, 2014, 4:59 p.m.)
Review request for Aurora, Kevin Sweeney and Maxim Khutornenko.
Repository: aurora
Description
---
In this change, i've added two outputs from thrift_wrapper_codegen.py:
- A service metadata file (java) that maps RPC method names to parameter and
type information
- A series of html documents that describes methods and types used in the API
(drawing from javadoc-style comments in api.thrift)
ApiBeta.java consumes the metadata file, and uses it to instruct gson how to
deserialize incoming requests. It in turn uses reflection to invoke the
appropriate method on the thrift interface implementation.
Diffs
-
build.gradle 751bde2f5052c66eac043f0b784c6e3bef9d3de3
src/main/java/org/apache/aurora/scheduler/http/ServletModule.java
27599f75603542069084631baf9195b8ad75e902
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
PRE-CREATION
src/main/python/apache/aurora/tools/java/thrift_wrapper_codegen.py
9d500baf82914733c0f92752d9570da009870aa9
src/main/thrift/org/apache/aurora/gen/api.thrift
8ee43fa1f0e2e699b0f1a321e673e49221b528ad
Diff: https://reviews.apache.org/r/23741/diff/
Testing
---
Manually walked through /apibeta in a web browser, all links seem working and
useful.
Started scheduler in vagrant, did some curling:
View the summary of all roles:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getRoleSummary | python
-mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
cronJobCount: 0,
jobCount: 1,
role: www-data
}
]
},
serverInfo: {
clusterName: example,
statsUrlPrefix: ,
thriftAPIVersion: 3
}
}
View the job summary for a role:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getJobSummary -d '{role:
www-data}' | python -mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
job: {
instanceCount: 3,
key: {
environment: devel,
name: hello_world,
role: www-data
},
owner: {
role: www-data,
user: vagrant
},
taskConfig: {
constraints: [
{
constraint: {
limit: 1
},
name: host
}
],
diskMb: 8,
environment: devel,
executorConfig: {
data: {\priority\: 0,
\health_check_config\: {\initial_interval_secs\: 15.0, \interval_secs\:
10.0, \timeout_secs\: 1.0, \max_consecutive_failures\: 0}, \name\:
\hello_world\, \environment\: \devel\, \max_task_failures\: 1,
\task\: {\processes\: [{\daemon\: false, \name\: \fetch_package\,
\ephemeral\: false, \max_failures\: 1, \min_duration\: 5, \cmdline\:
\cp /vagrant/hello_world.py . echo f182ae9a09cc7cea07dac9d595d5494a
chmod +x hello_world.py\, \final\: false}, {\daemon\: false, \name\:
\hello_world\, \ephemeral\: false, \max_failures\: 1, \min_duration\:
5, \cmdline\: \python hello_world.py\, \final\: false}], \name\:
\fetch_package\,
Re: Review Request 23741: Add a human-friendly HTTP/JSON API with auto-generated documentation.
On July 22, 2014, 10:52 p.m., David McLaughlin wrote:
Overall I think this is great. For my own curiosity - any reason why you've
used gson over the simple JSON protocol that comes with Thrift?
Bill Farner wrote:
TSimpleJSONProtocol is one-way, so it doesn't solve consumption of
parameters. I originally looked into writing my own TProtocol, but my gut
instinct would be that it would be a lot more code without much payoff.
Makes sense, thanks.
- David
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/#review48442
---
On July 21, 2014, 4:59 p.m., Bill Farner wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/
---
(Updated July 21, 2014, 4:59 p.m.)
Review request for Aurora, Kevin Sweeney and Maxim Khutornenko.
Repository: aurora
Description
---
In this change, i've added two outputs from thrift_wrapper_codegen.py:
- A service metadata file (java) that maps RPC method names to parameter and
type information
- A series of html documents that describes methods and types used in the API
(drawing from javadoc-style comments in api.thrift)
ApiBeta.java consumes the metadata file, and uses it to instruct gson how to
deserialize incoming requests. It in turn uses reflection to invoke the
appropriate method on the thrift interface implementation.
Diffs
-
build.gradle 751bde2f5052c66eac043f0b784c6e3bef9d3de3
src/main/java/org/apache/aurora/scheduler/http/ServletModule.java
27599f75603542069084631baf9195b8ad75e902
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
PRE-CREATION
src/main/python/apache/aurora/tools/java/thrift_wrapper_codegen.py
9d500baf82914733c0f92752d9570da009870aa9
src/main/thrift/org/apache/aurora/gen/api.thrift
8ee43fa1f0e2e699b0f1a321e673e49221b528ad
Diff: https://reviews.apache.org/r/23741/diff/
Testing
---
Manually walked through /apibeta in a web browser, all links seem working and
useful.
Started scheduler in vagrant, did some curling:
View the summary of all roles:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getRoleSummary | python
-mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
cronJobCount: 0,
jobCount: 1,
role: www-data
}
]
},
serverInfo: {
clusterName: example,
statsUrlPrefix: ,
thriftAPIVersion: 3
}
}
View the job summary for a role:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getJobSummary -d '{role:
www-data}' | python -mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
job: {
instanceCount: 3,
key: {
environment: devel,
name: hello_world,
role: www-data
},
owner: {
role: www-data,
user: vagrant
},
taskConfig: {
constraints: [
{
constraint: {
limit: 1
},
name: host
}
],
diskMb: 8,
environment: devel,
executorConfig: {
data: {\priority\: 0,
\health_check_config\: {\initial_interval_secs\: 15.0, \interval_secs\:
10.0, \timeout_secs\: 1.0, \max_consecutive_failures\: 0}, \name\:
\hello_world\, \environment\: \devel\, \max_task_failures\: 1,
\task\: {\processes\: [{\daemon\: false, \name\: \fetch_package\,
\ephemeral\: false, \max_failures\: 1, \min_duration\: 5, \cmdline\:
\cp /vagrant/hello_world.py . echo f182ae9a09cc7cea07dac9d595d5494a
chmod +x hello_world.py\, \final\: false}, {\daemon\: false, \name\:
\hello_world\, \ephemeral\: false, \max_failures\: 1, \min_duration\:
5, \cmdline\: \python hello_world.py\, \final\: false}], \name\:
Re: Review Request 23107: Database-backed implementation of SchedulerStore.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23107/#review48448 --- Ship it! Ship It! - Kevin Sweeney On July 1, 2014, 6:33 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23107/ --- (Updated July 1, 2014, 6:33 p.m.) Review request for Aurora and Kevin Sweeney. Bugs: AURORA-555 https://issues.apache.org/jira/browse/AURORA-555 Repository: aurora Description --- Database-backed implementation of SchedulerStore. Diffs - src/main/java/org/apache/aurora/scheduler/SchedulerLifecycle.java b4139eb5a6a9a281f7d2beaf04f7209b7c221980 src/main/java/org/apache/aurora/scheduler/storage/ForwardingStore.java 94f03f0d66291bbf919aea9f5149e49f8b4c38da src/main/java/org/apache/aurora/scheduler/storage/SchedulerStore.java 057a2e6a5f1c935eb0633cf5d97e582cd05d0c31 src/main/java/org/apache/aurora/scheduler/storage/db/DbLockStore.java f5ed139610382a0a9aa057ed0ae33576201d6767 src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 348527997ea89fd71952a9f1ec0c67c3087f5a61 src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java 0d8ad6108b061f7b3c39e3ab5ff08841d0dd2733 src/main/java/org/apache/aurora/scheduler/storage/db/FrameworkIdMapper.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/db/MigrationModule.java 77cd8abffca6bf20d753abe35ee212a7cf53668b src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java 1757ee3ba3b7f6fe18130294ed4f849bb0552e15 src/main/java/org/apache/aurora/scheduler/storage/mem/MemSchedulerStore.java a78b395b2b7621a8b1b24e3b6ad2a21ab1900d9a src/main/java/org/apache/aurora/scheduler/storage/mem/MemStorage.java cc206314d93ea7b160df7f809d072f90e23cc4b3 src/main/java/org/apache/aurora/scheduler/storage/mem/MemStorageModule.java 65565f1bb14dd4c6425f67f27ea06566d57b65cf src/main/resources/org/apache/aurora/scheduler/storage/db/FrameworkIdMapper.xml PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 3298eb38644b6fa7096801a69f8b88d0331ce4a7 src/test/java/org/apache/aurora/scheduler/SchedulerLifecycleTest.java 57095cea18936fb2ddf92bce587aea58e81996c8 src/test/java/org/apache/aurora/scheduler/storage/db/DbSchedulerStoreTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplTest.java 22a40481b5b246989c2e17a72a0325649382dde1 Diff: https://reviews.apache.org/r/23107/diff/ Testing --- ./gradlew build -Pq Thanks, Bill Farner
Re: Review Request 23741: Add a human-friendly HTTP/JSON API with auto-generated documentation.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/#review48449
---
Ship it!
lgtm.
- David McLaughlin
On July 21, 2014, 4:59 p.m., Bill Farner wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/
---
(Updated July 21, 2014, 4:59 p.m.)
Review request for Aurora, Kevin Sweeney and Maxim Khutornenko.
Repository: aurora
Description
---
In this change, i've added two outputs from thrift_wrapper_codegen.py:
- A service metadata file (java) that maps RPC method names to parameter and
type information
- A series of html documents that describes methods and types used in the API
(drawing from javadoc-style comments in api.thrift)
ApiBeta.java consumes the metadata file, and uses it to instruct gson how to
deserialize incoming requests. It in turn uses reflection to invoke the
appropriate method on the thrift interface implementation.
Diffs
-
build.gradle 751bde2f5052c66eac043f0b784c6e3bef9d3de3
src/main/java/org/apache/aurora/scheduler/http/ServletModule.java
27599f75603542069084631baf9195b8ad75e902
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
PRE-CREATION
src/main/python/apache/aurora/tools/java/thrift_wrapper_codegen.py
9d500baf82914733c0f92752d9570da009870aa9
src/main/thrift/org/apache/aurora/gen/api.thrift
8ee43fa1f0e2e699b0f1a321e673e49221b528ad
Diff: https://reviews.apache.org/r/23741/diff/
Testing
---
Manually walked through /apibeta in a web browser, all links seem working and
useful.
Started scheduler in vagrant, did some curling:
View the summary of all roles:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getRoleSummary | python
-mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
cronJobCount: 0,
jobCount: 1,
role: www-data
}
]
},
serverInfo: {
clusterName: example,
statsUrlPrefix: ,
thriftAPIVersion: 3
}
}
View the job summary for a role:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getJobSummary -d '{role:
www-data}' | python -mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
job: {
instanceCount: 3,
key: {
environment: devel,
name: hello_world,
role: www-data
},
owner: {
role: www-data,
user: vagrant
},
taskConfig: {
constraints: [
{
constraint: {
limit: 1
},
name: host
}
],
diskMb: 8,
environment: devel,
executorConfig: {
data: {\priority\: 0,
\health_check_config\: {\initial_interval_secs\: 15.0, \interval_secs\:
10.0, \timeout_secs\: 1.0, \max_consecutive_failures\: 0}, \name\:
\hello_world\, \environment\: \devel\, \max_task_failures\: 1,
\task\: {\processes\: [{\daemon\: false, \name\: \fetch_package\,
\ephemeral\: false, \max_failures\: 1, \min_duration\: 5, \cmdline\:
\cp /vagrant/hello_world.py . echo f182ae9a09cc7cea07dac9d595d5494a
chmod +x hello_world.py\, \final\: false}, {\daemon\: false, \name\:
\hello_world\, \ephemeral\: false, \max_failures\: 1, \min_duration\:
5, \cmdline\: \python hello_world.py\, \final\: false}], \name\:
\fetch_package\, \finalization_wait\: 30, \max_failures\: 1,
\max_concurrency\: 0, \resources\: {\disk\: 8388608, \ram\: 1048576,
\cpu\: 1.0}, \constraints\: [{\order\: [\fetch_package\,
\hello_world\]}]}, \enable_hooks\: false, \cluster
\: \devcluster\, \production\: false, \role\: \www-data\},
name: AuroraExecutor
},
isService: true,
Re: Review Request 23741: Add a human-friendly HTTP/JSON API with auto-generated documentation.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/#review48450
---
Ship it!
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
https://reviews.apache.org/r/23741/#comment85067
Should this location be hardcoded like this or can it be injected? Right
now I wouldn't expect removing ITaskConfig to break autogenerated documentation.
- Kevin Sweeney
On July 21, 2014, 9:59 a.m., Bill Farner wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/
---
(Updated July 21, 2014, 9:59 a.m.)
Review request for Aurora, Kevin Sweeney and Maxim Khutornenko.
Repository: aurora
Description
---
In this change, i've added two outputs from thrift_wrapper_codegen.py:
- A service metadata file (java) that maps RPC method names to parameter and
type information
- A series of html documents that describes methods and types used in the API
(drawing from javadoc-style comments in api.thrift)
ApiBeta.java consumes the metadata file, and uses it to instruct gson how to
deserialize incoming requests. It in turn uses reflection to invoke the
appropriate method on the thrift interface implementation.
Diffs
-
build.gradle 751bde2f5052c66eac043f0b784c6e3bef9d3de3
src/main/java/org/apache/aurora/scheduler/http/ServletModule.java
27599f75603542069084631baf9195b8ad75e902
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
PRE-CREATION
src/main/python/apache/aurora/tools/java/thrift_wrapper_codegen.py
9d500baf82914733c0f92752d9570da009870aa9
src/main/thrift/org/apache/aurora/gen/api.thrift
8ee43fa1f0e2e699b0f1a321e673e49221b528ad
Diff: https://reviews.apache.org/r/23741/diff/
Testing
---
Manually walked through /apibeta in a web browser, all links seem working and
useful.
Started scheduler in vagrant, did some curling:
View the summary of all roles:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getRoleSummary | python
-mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
cronJobCount: 0,
jobCount: 1,
role: www-data
}
]
},
serverInfo: {
clusterName: example,
statsUrlPrefix: ,
thriftAPIVersion: 3
}
}
View the job summary for a role:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getJobSummary -d '{role:
www-data}' | python -mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
job: {
instanceCount: 3,
key: {
environment: devel,
name: hello_world,
role: www-data
},
owner: {
role: www-data,
user: vagrant
},
taskConfig: {
constraints: [
{
constraint: {
limit: 1
},
name: host
}
],
diskMb: 8,
environment: devel,
executorConfig: {
data: {\priority\: 0,
\health_check_config\: {\initial_interval_secs\: 15.0, \interval_secs\:
10.0, \timeout_secs\: 1.0, \max_consecutive_failures\: 0}, \name\:
\hello_world\, \environment\: \devel\, \max_task_failures\: 1,
\task\: {\processes\: [{\daemon\: false, \name\: \fetch_package\,
\ephemeral\: false, \max_failures\: 1, \min_duration\: 5, \cmdline\:
\cp /vagrant/hello_world.py . echo f182ae9a09cc7cea07dac9d595d5494a
chmod +x hello_world.py\, \final\: false}, {\daemon\: false, \name\:
\hello_world\, \ephemeral\: false, \max_failures\: 1, \min_duration\:
5, \cmdline\: \python hello_world.py\, \final\: false}], \name\:
\fetch_package\, \finalization_wait\: 30, \max_failures\: 1,
\max_concurrency\: 0, \resources\: {\disk\: 8388608, \ram\: 1048576,
\cpu\: 1.0}, \constraints\: [{\order\: [\fetch_package\,
Re: Review Request 23776: Add --stacktrace to gradle build command.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23776/#review48451 --- Ship it! Ship It! - Kevin Sweeney On July 21, 2014, 3:27 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23776/ --- (Updated July 21, 2014, 3:27 p.m.) Review request for Aurora and Kevin Sweeney. Repository: aurora Description --- I'm hoping this gives more insight into some recent build flakiness, specifically this error: net.rubygrapefruit.platform.internal.jni.PosixFileFunctions.symlink(Ljava/lang/String;Ljava/lang/String;Lnet/rubygrapefruit/platform/internal/FunctionResult;)V This appears to be related to GRADLE-3076 [1], but --stacktrace may give us more information. [1] http://issues.gradle.org/browse/GRADLE-3076 Diffs - build-support/jenkins/build.sh 466aae208659cb08d633c12b674a0c9ec2819bba Diff: https://reviews.apache.org/r/23776/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 23471: add support for Mesos masters running with --no-root_submissions, see AURORA-311
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23471/#review48453 --- Ship it! Ship It! - Kevin Sweeney On July 17, 2014, 11:11 a.m., Bjoern Metzdorf wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23471/ --- (Updated July 17, 2014, 11:11 a.m.) Review request for Aurora and Kevin Sweeney. Bugs: AURORA-311 https://issues.apache.org/jira/browse/AURORA-311 Repository: aurora Description --- Review board entry for https://issues.apache.org/jira/browse/AURORA-311 Diffs - src/main/java/org/apache/aurora/scheduler/DriverFactory.java db864a0 Diff: https://reviews.apache.org/r/23471/diff/ Testing --- Tested successfully on internal cluster with version 0.5.1. Aurora was able to connect to a mesos master that was running with --no-root_submissions and was able to schedule tasks with a role of EXECUTOR_USER that then were running as the specified executor user. Thanks, Bjoern Metzdorf
Re: Review Request 23471: add support for Mesos masters running with --no-root_submissions, see AURORA-311
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23471/ --- (Updated July 22, 2014, 4:28 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Changes --- +wfarner for a second set of committer eyes Bugs: AURORA-311 https://issues.apache.org/jira/browse/AURORA-311 Repository: aurora Description --- Review board entry for https://issues.apache.org/jira/browse/AURORA-311 Diffs - src/main/java/org/apache/aurora/scheduler/DriverFactory.java db864a0 Diff: https://reviews.apache.org/r/23471/diff/ Testing --- Tested successfully on internal cluster with version 0.5.1. Aurora was able to connect to a mesos master that was running with --no-root_submissions and was able to schedule tasks with a role of EXECUTOR_USER that then were running as the specified executor user. Thanks, Bjoern Metzdorf
Re: Review Request 23471: add support for Mesos masters running with --no-root_submissions, see AURORA-311
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23471/ --- (Updated July 22, 2014, 4:28 p.m.) Review request for Aurora, Kevin Sweeney and Brian Wickman. Changes --- -wfarner, +wickman Bugs: AURORA-311 https://issues.apache.org/jira/browse/AURORA-311 Repository: aurora Description --- Review board entry for https://issues.apache.org/jira/browse/AURORA-311 Diffs - src/main/java/org/apache/aurora/scheduler/DriverFactory.java db864a0 Diff: https://reviews.apache.org/r/23471/diff/ Testing --- Tested successfully on internal cluster with version 0.5.1. Aurora was able to connect to a mesos master that was running with --no-root_submissions and was able to schedule tasks with a role of EXECUTOR_USER that then were running as the specified executor user. Thanks, Bjoern Metzdorf
Re: Review Request 23832: Fix constraint violation caused by slave ID change.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23832/#review48464 --- Ship it! Ship It! - Maxim Khutornenko On July 23, 2014, 1:25 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23832/ --- (Updated July 23, 2014, 1:25 a.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-596 https://issues.apache.org/jira/browse/AURORA-596 Repository: aurora Description --- Fix constraint violation caused by slave ID change. Diffs - src/main/java/org/apache/aurora/scheduler/storage/db/DbAttributeStore.java c5bfa1b6ac2fab736e1a75ddb4faf7d4b24fc9db src/main/resources/org/apache/aurora/scheduler/storage/db/AttributeMapper.xml f9b689dab94a14a1c4686090b246683969dc4c59 src/test/java/org/apache/aurora/scheduler/storage/db/DbAttributeStoreTest.java 3962fb0e70773d00412205f3f882f4d5137866f8 Diff: https://reviews.apache.org/r/23832/diff/ Testing --- ./gradlew clean build -Pq Thanks, Bill Farner
Review Request 23834: Refactoring SchedulerCore (restartShards)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23834/ --- Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-94 https://issues.apache.org/jira/browse/AURORA-94 Repository: aurora Description --- Moving restartShards from SchedulerCore into SchedulerThriftInterface. Diffs - src/main/java/org/apache/aurora/scheduler/state/SchedulerCore.java 137bd6ccafc48508ccd48004e7111fe69449152d src/main/java/org/apache/aurora/scheduler/state/SchedulerCoreImpl.java adc2dbc8c451260e8e4749dd07f47e1b0cd7dda2 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 7f94e0447066a2e9f2e460a284c7f93eac347a91 src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 5ec7cf395080cd593a5d5291eff835f89c65dac2 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java d3ac7c7585e8fcac6d26f06f53c9101aa9787808 Diff: https://reviews.apache.org/r/23834/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 23834: Refactoring SchedulerCore (restartShards)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23834/#review48467 --- Ship it! src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java https://reviews.apache.org/r/23834/#comment85095 @Nullable - Bill Farner On July 23, 2014, 2:21 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23834/ --- (Updated July 23, 2014, 2:21 a.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-94 https://issues.apache.org/jira/browse/AURORA-94 Repository: aurora Description --- Moving restartShards from SchedulerCore into SchedulerThriftInterface. Diffs - src/main/java/org/apache/aurora/scheduler/state/SchedulerCore.java 137bd6ccafc48508ccd48004e7111fe69449152d src/main/java/org/apache/aurora/scheduler/state/SchedulerCoreImpl.java adc2dbc8c451260e8e4749dd07f47e1b0cd7dda2 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 7f94e0447066a2e9f2e460a284c7f93eac347a91 src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 5ec7cf395080cd593a5d5291eff835f89c65dac2 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java d3ac7c7585e8fcac6d26f06f53c9101aa9787808 Diff: https://reviews.apache.org/r/23834/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 23834: Refactoring SchedulerCore (restartShards)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23834/ --- (Updated July 23, 2014, 2:36 a.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Changes --- CR comments. Bugs: AURORA-94 https://issues.apache.org/jira/browse/AURORA-94 Repository: aurora Description --- Moving restartShards from SchedulerCore into SchedulerThriftInterface. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/state/SchedulerCore.java 137bd6ccafc48508ccd48004e7111fe69449152d src/main/java/org/apache/aurora/scheduler/state/SchedulerCoreImpl.java adc2dbc8c451260e8e4749dd07f47e1b0cd7dda2 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 7f94e0447066a2e9f2e460a284c7f93eac347a91 src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 5ec7cf395080cd593a5d5291eff835f89c65dac2 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java d3ac7c7585e8fcac6d26f06f53c9101aa9787808 Diff: https://reviews.apache.org/r/23834/diff/ Testing --- gradle -Pq build Thanks, Maxim Khutornenko
Re: Review Request 23741: Add a human-friendly HTTP/JSON API with auto-generated documentation.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/#review48327
---
Ship it!
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
https://reviews.apache.org/r/23741/#comment85099
Since metadata is extracted using method name, would it make sense to wrap
this into a LoadingCache? Might save a bit of perf by not going through class
metadata every time a method is called.
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
https://reviews.apache.org/r/23741/#comment84816
If you decide to keep it move it into constant?
- Maxim Khutornenko
On July 21, 2014, 4:59 p.m., Bill Farner wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23741/
---
(Updated July 21, 2014, 4:59 p.m.)
Review request for Aurora, Kevin Sweeney and Maxim Khutornenko.
Repository: aurora
Description
---
In this change, i've added two outputs from thrift_wrapper_codegen.py:
- A service metadata file (java) that maps RPC method names to parameter and
type information
- A series of html documents that describes methods and types used in the API
(drawing from javadoc-style comments in api.thrift)
ApiBeta.java consumes the metadata file, and uses it to instruct gson how to
deserialize incoming requests. It in turn uses reflection to invoke the
appropriate method on the thrift interface implementation.
Diffs
-
build.gradle 751bde2f5052c66eac043f0b784c6e3bef9d3de3
src/main/java/org/apache/aurora/scheduler/http/ServletModule.java
27599f75603542069084631baf9195b8ad75e902
src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java
PRE-CREATION
src/main/python/apache/aurora/tools/java/thrift_wrapper_codegen.py
9d500baf82914733c0f92752d9570da009870aa9
src/main/thrift/org/apache/aurora/gen/api.thrift
8ee43fa1f0e2e699b0f1a321e673e49221b528ad
Diff: https://reviews.apache.org/r/23741/diff/
Testing
---
Manually walked through /apibeta in a web browser, all links seem working and
useful.
Started scheduler in vagrant, did some curling:
View the summary of all roles:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getRoleSummary | python
-mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
cronJobCount: 0,
jobCount: 1,
role: www-data
}
]
},
serverInfo: {
clusterName: example,
statsUrlPrefix: ,
thriftAPIVersion: 3
}
}
View the job summary for a role:
$ curl -s -X POST http://192.168.33.7:8081/apibeta/getJobSummary -d '{role:
www-data}' | python -mjson.tool
{
DEPRECATEDversion: {
major: 3
},
details: [
{
message: Welcome, beta API user! Please help us find and fix
the bugs that likely lurk in this API!
}
],
responseCode: OK,
result: {
summaries: [
{
job: {
instanceCount: 3,
key: {
environment: devel,
name: hello_world,
role: www-data
},
owner: {
role: www-data,
user: vagrant
},
taskConfig: {
constraints: [
{
constraint: {
limit: 1
},
name: host
}
],
diskMb: 8,
environment: devel,
executorConfig: {
data: {\priority\: 0,
\health_check_config\: {\initial_interval_secs\: 15.0, \interval_secs\:
10.0, \timeout_secs\: 1.0, \max_consecutive_failures\: 0}, \name\:
\hello_world\, \environment\: \devel\, \max_task_failures\: 1,
\task\: {\processes\: [{\daemon\: false, \name\: \fetch_package\,
\ephemeral\: false, \max_failures\: 1, \min_duration\: 5, \cmdline\:
\cp /vagrant/hello_world.py . echo f182ae9a09cc7cea07dac9d595d5494a
chmod +x hello_world.py\, \final\: false}, {\daemon\: false, \name\:
\hello_world\, \ephemeral\: false, \max_failures\: 1, \min_duration\:
5, \cmdline\: \python hello_world.py\,
