Re: Review Request 37956: Only process instances if there's been a change.

2015-09-02 Thread David McLaughlin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37956/#review97500
---

Ship it!


Ship It!

- David McLaughlin


On Sept. 1, 2015, 4:03 p.m., Joshua Cohen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37956/
> ---
> 
> (Updated Sept. 1, 2015, 4:03 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Bill Farner.
> 
> 
> Bugs: AURORA-1345
> https://issues.apache.org/jira/browse/AURORA-1345
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> The `true` parameter to `scope.$watch` tells Angular to only execute the 
> supplied function if the watched data has changed. Given our 15s polling 
> interval is likely less than time it takes for any progress to be made on an 
> update, this removes the majority of (no-op) DOM operations.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/js/directives.js 
> d8f722c441b9399e7eacd6ae29966b3fb24a422d 
> 
> Diff: https://reviews.apache.org/r/37956/diff/
> 
> 
> Testing
> ---
> 
> Left tab open continually polling a job with ~3000 instances, confirmed that 
> memory usage did not grow beyond ~200MB.
> 
> 
> Thanks,
> 
> Joshua Cohen
> 
>

Re: Review Request 37818: Moved executor settings configuration to loadable JSON

2015-09-02 Thread Bill Farner 


> On Aug. 31, 2015, 7:35 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, line 1
> > 
> >
> > Can you expand this example to include the command executor?  That's 
> > likely to be the first non-default executor folks will want to try.
> 
> Renan DelValle wrote:
> Sure thing. Although, be advised that the functionality for custom 
> executors is not included in this patch. This patch only lays the ground 
> work. The next patch will include the ability to actually select them and run 
> them.

Cool, that's actually what i wanted to see - a format change.  If we want the 
format to support multiple executors, we should use that format now so it works 
once multiple executors _are_ supported.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97231
---


On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> ---
> 
> (Updated Sept. 1, 2015, 7:35 p.m.)
> 
> 
> Review request for Aurora.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is the first stage in a series of patches to create support for custom 
> executors. In an effort to expedite the review process, I have decided to 
> break down my patch into multiple pieces that when/if commited won't break 
> the trunk.
> 
> This patch includes the ability to load configuration from a JSON file. A 
> JSON example file is included in examples/vagrant/executors-config.json
> 
> Command line arguments have been eliminated and moved over to the JSON file. 
> GSON is leveraged and does most of the work with the aid of a few custom 
> deserializers that were needed. 
> 
> Note that right now a global container mount that does not follow 
> specification will cause the scheduler to detect the error an exit early. It 
> is up for discussion if this is the desired behavior or if we should just 
> ignore said mount.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/executors-config.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
> 4f43892723db4744db205ea7dd107e9e9ce9d5db 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 8047622e206c9827e5cd8e40152a278d495bd0ff 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>  PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> 1a25924d789295c5950947f5e302e1d1fbec68f2 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java
>  PRE-CREATION 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json
>  PRE-CREATION 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json
>  PRE-CREATION 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json
>  PRE-CREATION 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/37818/diff/
> 
> 
> Testing
> ---
> 
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a 
> flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> 
> Thanks,
> 
> Renan DelValle
> 
>

Re: Review Request 37926: Adding oversubscription e2e tests.

2015-09-02 Thread Maxim Khutornenko 


> On Aug. 31, 2015, 11:31 p.m., Bill Farner wrote:
> > src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh, line 322
> > 
> >
> > We probably only want to create a job using revocable resources and 
> > watch that it doesn't explode.  The full extra round of test cases seems 
> > excessive.
> 
> Maxim Khutornenko wrote:
> I started that route but then quickly realized I wanted observer tests, 
> make sure update works and etc. Seems like too much customization for no 
> apparent benefit. Are you concerned about the overall execution speed?
> 
> Bill Farner wrote:
> Yeah, i'm worried about execution speed if we're concerned about covering 
> this full set of behavior for arbitrary features.  For example, i'm not 
> convinced that job updates need to be exercised for this case (or checking 
> that the oberver doesn't 404, for that matter).  I would much rather see 
> feature-related test cases be as feature-specific as possible.

I see your point. Carved out creation and observer tests only to ensure 
revocable tasks can get fully runnable.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37926/#review97195
---


On Sept. 1, 2015, 5:15 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37926/
> ---
> 
> (Updated Sept. 1, 2015, 5:15 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Also, had to backout client tier validation added earlier until the feature 
> exits beta.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   examples/vagrant/upstart/mesos-slave.conf 
> 2b6a60673fc0a7ea3b73471701cd5d3efd6ce639 
>   src/main/java/org/apache/aurora/scheduler/http/Offers.java 
> 4329ce10189df41a7a45cd6cc369333d7d5f65b0 
>   src/main/python/apache/aurora/client/config.py 
> f5ac084ae5b448221363add9476334742b217b92 
>   src/test/python/apache/aurora/client/test_config.py 
> 37459f5218e76fac831601806a0331927c0ee2c0 
>   src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 
> c1a10d8ea60be6aa56e4517fb34288d7d5ae1480 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 
> 870b3e68035fdf86253cf9b92b606645134b3369 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora 
> e55aad3a58d4e3c19332e06b70771f51f07aa9b7 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 
> 423dd4d4e8b03c2f852e25acd9340bd6288b7d24 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> 018efbe43de0c829799bc2f789958c55b1cb61a4 
> 
> Diff: https://reviews.apache.org/r/37926/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 37818: Moved executor settings configuration to loadable JSON

2015-09-02 Thread Bill Farner 


> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java,
> >  line 87
> > 
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on 
> > ExecutorSettings)
> > - avoid hand-rolling deserializers
> > 
> > This sounds like more work, but it's not that bad.  The major 
> > difference is that you'll create a class in this file that defines the JSON 
> > structure, and does so in a way that the json deserializer can work with.
> > 
> > Maxim just did something very similar, which you can crib from:
> > 
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > 
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55

Oh, the detail i neglected to mention - i suggest you then write code to 
translate your config schema object into ExecutorSettings.  This can provides 
compile-time guarantees that we don't have with the current code, making it 
easier to maintain.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
---


On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> ---
> 
> (Updated Sept. 1, 2015, 7:35 p.m.)
> 
> 
> Review request for Aurora.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is the first stage in a series of patches to create support for custom 
> executors. In an effort to expedite the review process, I have decided to 
> break down my patch into multiple pieces that when/if commited won't break 
> the trunk.
> 
> This patch includes the ability to load configuration from a JSON file. A 
> JSON example file is included in examples/vagrant/executors-config.json
> 
> Command line arguments have been eliminated and moved over to the JSON file. 
> GSON is leveraged and does most of the work with the aid of a few custom 
> deserializers that were needed. 
> 
> Note that right now a global container mount that does not follow 
> specification will cause the scheduler to detect the error an exit early. It 
> is up for discussion if this is the desired behavior or if we should just 
> ignore said mount.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/executors-config.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
> 4f43892723db4744db205ea7dd107e9e9ce9d5db 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 8047622e206c9827e5cd8e40152a278d495bd0ff 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>  PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> 1a25924d789295c5950947f5e302e1d1fbec68f2 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java
>  PRE-CREATION 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json
>  PRE-CREATION 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json
>  PRE-CREATION 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json
>  PRE-CREATION 
>   
>

Re: Review Request 37818: Moved executor settings configuration to loadable JSON

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
---



examples/vagrant/executors-config.json (lines 4 - 7)


The code later converts this array into a single command string.  I suggest 
we just make this a string here (and save some code down the line).



examples/vagrant/executors-config.json (lines 17 - 18)


Can you omit this since it's blank?



examples/vagrant/executors-config.json (line 32)


Use single quotes to avoid escaping:

"'Hello World from Aurora!'"



src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 


Please add an entry in /NEWS under 0.10.0 to inform people what args were 
removed, and what the replacement is.



src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java (line 166)


throw Throwables.propagate(e);

Otherwise the app will exit in a non-obvious way.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java (lines 61 - 63)


remove



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java (lines 95 - 96)


I suggest removing this for now.  There's a lot of ways fetching the URIs 
and executing commands can fail.  For the sake of simplicity, let's allow them 
to fail and focus on providing good visibility into the failure.



src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
 (line 46)


please add a javadoc for this class



src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
 (line 62)


Save a few lines:

```
String config = Files.toString(configFile, StandardCharsets.UTF_8);
```



src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
 (lines 66 - 71)


You might want to consider using jackson instead of json.  We recently 
learned that gson will silently ignore unexpected json properties, which is not 
behavior we want for a config file.  This doesn't change much for you, see 
pointers in comment below for an example.



src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
 (line 79)


remove the ```\n```, same with the string below



src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
 (line 87)


This code will be easier to maintain long-term with 2 goals in mind:
- keep json-related code within this class (no annotations on 
ExecutorSettings)
- avoid hand-rolling deserializers

This sounds like more work, but it's not that bad.  The major difference is 
that you'll create a class in this file that defines the JSON structure, and 
does so in a way that the json deserializer can work with.

Maxim just did something very similar, which you can crib from:

https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147

https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55


- Bill Farner


On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> ---
> 
> (Updated Sept. 1, 2015, 7:35 p.m.)
> 
> 
> Review request for Aurora.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is the first stage in a series of patches to create support for custom 
> executors. In an effort to expedite the review process, I have decided to 
> break down my patch into multiple pieces that when/if commited won't break 
> the trunk.
> 
> This patch includes the ability to load configuration from a JSON file. A 
> JSON example file is included in examples/vagrant/executors-config.json
> 
> Command line arguments have been eliminated and moved over to the JSON file. 
> GSON is leveraged and does most of the work with the aid of a few custom 
> deserializers that were needed. 
> 
> Note that right now a global container mount that does not follow 
> specification will cause the scheduler to detect the

Re: Review Request 37926: Adding oversubscription e2e tests.

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37926/#review97480
---



src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh (line 255)


What's being tested here?  Are you trying to determine whether the task 
executed correctly?  If so, maybe you should watch for it to enter RUNNING 
instead?


- Bill Farner


On Sept. 2, 2015, 9:45 a.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37926/
> ---
> 
> (Updated Sept. 2, 2015, 9:45 a.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Also, had to backout client tier validation added earlier until the feature 
> exits beta.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   examples/vagrant/upstart/mesos-slave.conf 
> 2b6a60673fc0a7ea3b73471701cd5d3efd6ce639 
>   src/main/java/org/apache/aurora/scheduler/http/Offers.java 
> 4329ce10189df41a7a45cd6cc369333d7d5f65b0 
>   src/main/python/apache/aurora/client/config.py 
> f5ac084ae5b448221363add9476334742b217b92 
>   src/test/python/apache/aurora/client/test_config.py 
> 37459f5218e76fac831601806a0331927c0ee2c0 
>   src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 
> c1a10d8ea60be6aa56e4517fb34288d7d5ae1480 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 
> 870b3e68035fdf86253cf9b92b606645134b3369 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora 
> e55aad3a58d4e3c19332e06b70771f51f07aa9b7 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 
> 423dd4d4e8b03c2f852e25acd9340bd6288b7d24 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> 018efbe43de0c829799bc2f789958c55b1cb61a4 
> 
> Diff: https://reviews.apache.org/r/37926/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 37956: Only process instances if there's been a change.

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37956/#review97479
---

Ship it!


Ship It!

- Bill Farner


On Sept. 1, 2015, 9:03 a.m., Joshua Cohen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37956/
> ---
> 
> (Updated Sept. 1, 2015, 9:03 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Bill Farner.
> 
> 
> Bugs: AURORA-1345
> https://issues.apache.org/jira/browse/AURORA-1345
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> The `true` parameter to `scope.$watch` tells Angular to only execute the 
> supplied function if the watched data has changed. Given our 15s polling 
> interval is likely less than time it takes for any progress to be made on an 
> update, this removes the majority of (no-op) DOM operations.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/js/directives.js 
> d8f722c441b9399e7eacd6ae29966b3fb24a422d 
> 
> Diff: https://reviews.apache.org/r/37956/diff/
> 
> 
> Testing
> ---
> 
> Left tab open continually polling a job with ~3000 instances, confirmed that 
> memory usage did not grow beyond ~200MB.
> 
> 
> Thanks,
> 
> Joshua Cohen
> 
>

Re: Review Request 37926: Adding oversubscription e2e tests.

2015-09-02 Thread Zameer Manji 


> On Sept. 2, 2015, 9:53 a.m., Bill Farner wrote:
> > src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh, line 255
> > 
> >
> > What's being tested here?  Are you trying to determine whether the task 
> > executed correctly?  If so, maybe you should watch for it to enter RUNNING 
> > instead?
> 
> Maxim Khutornenko wrote:
> This is our test equivalent of "aurora job create --wait-until=RUNNING". 
> It's actually more suitable for e2e purpose as it has timeout whereas 
> --wait-until blocks forever. This is especially important in view of bugs 
> like AURORA-1459 where tasks never reach RUNNING.
> 
> Bill Farner wrote:
> Certainly a hack, but works for me - still better than the previous 
> iteration.

AFAIK, I added test_observer_ui to ensure the observer can load checkpoints for 
the task. I guess it also serves the `aurora job create --wait-until=RUNNING` 
case as well.


- Zameer


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37926/#review97480
---


On Sept. 2, 2015, 9:45 a.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37926/
> ---
> 
> (Updated Sept. 2, 2015, 9:45 a.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Also, had to backout client tier validation added earlier until the feature 
> exits beta.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   examples/vagrant/upstart/mesos-slave.conf 
> 2b6a60673fc0a7ea3b73471701cd5d3efd6ce639 
>   src/main/java/org/apache/aurora/scheduler/http/Offers.java 
> 4329ce10189df41a7a45cd6cc369333d7d5f65b0 
>   src/main/python/apache/aurora/client/config.py 
> f5ac084ae5b448221363add9476334742b217b92 
>   src/test/python/apache/aurora/client/test_config.py 
> 37459f5218e76fac831601806a0331927c0ee2c0 
>   src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 
> c1a10d8ea60be6aa56e4517fb34288d7d5ae1480 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 
> 870b3e68035fdf86253cf9b92b606645134b3369 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora 
> e55aad3a58d4e3c19332e06b70771f51f07aa9b7 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 
> 423dd4d4e8b03c2f852e25acd9340bd6288b7d24 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> 018efbe43de0c829799bc2f789958c55b1cb61a4 
> 
> Diff: https://reviews.apache.org/r/37926/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 37818: Moved executor settings configuration to loadable JSON

2015-09-02 Thread Bill Farner 


> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java,
> >  line 87
> > 
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on 
> > ExecutorSettings)
> > - avoid hand-rolling deserializers
> > 
> > This sounds like more work, but it's not that bad.  The major 
> > difference is that you'll create a class in this file that defines the JSON 
> > structure, and does so in a way that the json deserializer can work with.
> > 
> > Maxim just did something very similar, which you can crib from:
> > 
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > 
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
> 
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to 
> translate your config schema object into ExecutorSettings.  This can provides 
> compile-time guarantees that we don't have with the current code, making it 
> easier to maintain.
> 
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm 
> looking through what Maxim did and I'll definitely crib here and there but I 
> can't find anything related to getting this done at compile time.

Don't be thrown by the mention of compile time, i'm just referring to refactor 
safety if/when the structures changed.  The concise way to put it is - use a 
different class for your config domain object.  This way you don't need to 
change ExecutorSettings to compensate for the fact that some code in another 
part of the system is using a json mapper.

Start by defining a class that matches your config schema:
```
class ExecutorConfig {
  String command;
  List resource;
  ...
}

class Resource {
  String value;  // note - i would like to see this renamed to uri
  boolean executable;
  boolean extract;
  boolean cache;
}
...
```
(that's just a quick example, feel free to adjust as you see fit)

Jackson is very similar to gson, there's just a behavior nuance that is 
desirable.  You'll use an `ObjectMapper` to parse the json to an 
`ExecutorConfig`, then write a function to transform your `ExecutorConfig` into 
`ExecutorSettings`.

```
private static ExecutorSettings configToSettings(ExecutorConfig config) {
  return new ExecutorSettings(...);
}
```


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
---


On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> ---
> 
> (Updated Sept. 1, 2015, 7:35 p.m.)
> 
> 
> Review request for Aurora.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is the first stage in a series of patches to create support for custom 
> executors. In an effort to expedite the review process, I have decided to 
> break down my patch into multiple pieces that when/if commited won't break 
> the trunk.
> 
> This patch includes the ability to load configuration from a JSON file. A 
> JSON example file is included in examples/vagrant/executors-config.json
> 
> Command line arguments have been eliminated and moved over to the JSON file. 
> GSON is leveraged and does most of the work with the aid of a few custom 
> deserializers that were needed. 
> 
> Note that right now a global container mount that does not follow 
> specification will cause the scheduler to detect the error an exit early. It 
> is up for discussion if this is the desired behavior or if we should just 
> ignore said mount.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/executors-config.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
> 4f43892723db4744db205ea7dd107e9e9ce9d5db 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 8047622e206c9827e5cd8e40152a278d495bd0ff 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   
>

Re: Review Request 38080: Remove openjdk runtime requirement.

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38080/#review97573
---

Ship it!


Ship It!

- Bill Farner


On Sept. 2, 2015, 4:36 p.m., Kevin Sweeney wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38080/
> ---
> 
> (Updated Sept. 2, 2015, 4:36 p.m.)
> 
> 
> Review request for Aurora, Steve Salevan and Bill Farner.
> 
> 
> Repository: aurora-packaging
> 
> 
> Description
> ---
> 
> Remove openjdk runtime requirement.
> 
> This addresses one of my +0s.
> 
> 
> Diffs
> -
> 
>   specs/rpm/aurora.spec a559b42d73a4211891d8c7569ea7ab2a3a6993ae 
> 
> Diff: https://reviews.apache.org/r/38080/diff/
> 
> 
> Testing
> ---
> 
> Using a clean vagrant environment installed aurora, then installed the oracle 
> jdk, then uninstalled java-1.8.0-openjdk-headless (that aurora brought in 
> transitively). RPM didn't complain and when I restarted the scheduler it was 
> running under the new JVM.
> 
> ```
> [vagrant@localhost ~]$ curl -s http://localhost:8081/vars | grep 
> 'jvm_prop_java_vendor '
> jvm_prop_java_vendor Oracle Corporation
> [vagrant@localhost ~]$ 
> ```
> 
> 
> Thanks,
> 
> Kevin Sweeney
> 
>

Re: Review Request 38080: Remove openjdk runtime requirement.

2015-09-02 Thread Bill Farner 


> On Sept. 2, 2015, 5:08 p.m., Bill Farner wrote:
> > Ship It!
> 
> Kevin Sweeney wrote:
> procedural question - I see there's a long-running 0.9.x branch - should 
> I merge this to master then merge this into that branch?

More or less, yes - i've been cherry-picking back onto 0.9.x.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38080/#review97573
---


On Sept. 2, 2015, 4:36 p.m., Kevin Sweeney wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38080/
> ---
> 
> (Updated Sept. 2, 2015, 4:36 p.m.)
> 
> 
> Review request for Aurora, Steve Salevan and Bill Farner.
> 
> 
> Repository: aurora-packaging
> 
> 
> Description
> ---
> 
> Remove openjdk runtime requirement.
> 
> This addresses one of my +0s.
> 
> 
> Diffs
> -
> 
>   specs/rpm/aurora.spec a559b42d73a4211891d8c7569ea7ab2a3a6993ae 
> 
> Diff: https://reviews.apache.org/r/38080/diff/
> 
> 
> Testing
> ---
> 
> Using a clean vagrant environment installed aurora, then installed the oracle 
> jdk, then uninstalled java-1.8.0-openjdk-headless (that aurora brought in 
> transitively). RPM didn't complain and when I restarted the scheduler it was 
> running under the new JVM.
> 
> ```
> [vagrant@localhost ~]$ curl -s http://localhost:8081/vars | grep 
> 'jvm_prop_java_vendor '
> jvm_prop_java_vendor Oracle Corporation
> [vagrant@localhost ~]$ 
> ```
> 
> 
> Thanks,
> 
> Kevin Sweeney
> 
>

Review Request 38081: Dropping quota check for dedicated jobs and exposing dedicated consumption.

2015-09-02 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38081/
---

Review request for Aurora, Joshua Cohen, Bill Farner, and Zameer Manji.


Bugs: AURORA-1460
https://issues.apache.org/jira/browse/AURORA-1460


Repository: aurora


Description
---

This change drops quota enforcement for dedicated jobs and adds dedicated 
resource consumption into the thrift API. Feel free to pushback on the latter 
but I think users will find it immediately confusing when we change the "Prod 
consumption" to something like "Prod shared pool consumption" in the UI/client 
and not provide the missing dedicated rollup numbers.


Diffs
-

  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
21137bbbdc3010c6b1e2cc0ebb3b99bfa8490563 
  src/main/java/org/apache/aurora/scheduler/quota/QuotaInfo.java 
b40ecd0af7c1d1bb9372bd89c741622ce4c9040c 
  src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 
8787aeaa6655cfab1e0a6d5719f9e08a89df7631 
  src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
3d89e43659750de63d7588f8574e7a350caea04b 
  src/main/python/apache/aurora/client/api/quota_check.py 
75406ac1d3ec9ba655daf7c6125f208f74290cfb 
  src/main/python/apache/aurora/client/cli/quota.py 
e8aa010f5ecce0464a8ad6b072bccba589fe16d7 
  src/main/resources/scheduler/assets/js/controllers.js 
98920196db34f2eb4dcad93429274517e7383efe 
  src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 
db60cd21d06d636505202bac7277a13dc24d46e6 
  
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 
4d4e752088f7dca99675cc66782ae046bbd516d6 
  src/test/python/apache/aurora/admin/test_admin.py 
d793293acc7f77c0081968334c38b984d865fbc8 
  src/test/python/apache/aurora/client/api/test_quota_check.py 
6c9bc373d1868bd6e0cae2f2218261f39fd7ab8f 
  src/test/python/apache/aurora/client/cli/test_quota.py 
3573e4c4575577e2232d2b99ca781b06a03d48d7 

Diff: https://reviews.apache.org/r/38081/diff/


Testing
---


Thanks,

Maxim Khutornenko

Review Request 38080: Remove openjdk runtime requirement.

2015-09-02 Thread Kevin Sweeney 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38080/
---

Review request for Aurora, Steve Salevan and Bill Farner.


Repository: aurora-packaging


Description
---

Remove openjdk runtime requirement.

This addresses one of my +0s.


Diffs
-

  specs/rpm/aurora.spec a559b42d73a4211891d8c7569ea7ab2a3a6993ae 

Diff: https://reviews.apache.org/r/38080/diff/


Testing
---

Using a clean vagrant environment installed aurora, then installed the oracle 
jdk, then uninstalled java-1.8.0-openjdk-headless (that aurora brought in 
transitively). RPM didn't complain and when I restarted the scheduler it was 
running under the new JVM.

```
[vagrant@localhost ~]$ curl -s http://localhost:8081/vars | grep 
'jvm_prop_java_vendor '
jvm_prop_java_vendor Oracle Corporation
[vagrant@localhost ~]$ 
```


Thanks,

Kevin Sweeney

Review Request 38084: Builder should run createrepo

2015-09-02 Thread Kevin Sweeney 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38084/
---

Review request for Aurora and Bill Farner.


Repository: aurora-packaging


Description
---

Builder should run createrepo


Diffs
-

  builder/rpm/centos-7/build.sh 912f7c788be648f8f3a11ec868a56d2994c62117 

Diff: https://reviews.apache.org/r/38084/diff/


Testing
---

Tested on 0.9.x branch


Thanks,

Kevin Sweeney

Re: Review Request 38079: Alter code generator to replace null collection fields with an empty collection.

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38079/#review97591
---


-1, don't review this - the generated code is broken.  Will fix/update.

- Bill Farner


On Sept. 2, 2015, 6:08 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38079/
> ---
> 
> (Updated Sept. 2, 2015, 6:08 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-1476
> https://issues.apache.org/jira/browse/AURORA-1476
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> We encountered this bug because tasks containing `null` collections would 
> return from the database with an empty set/list.  We could debate over 
> whether `null` is preferable, but ultimately we need to force consistent use 
> of an empty collection or `null` to make this work across the board.
> 
> The specific issue in this case is that we try to share relations to entries 
> in the `task_configs` table by querying to find a pre-existing entry with the 
> same contents.  We use object equality to determine this.  That approach is 
> problematic when the 'same' content from thrift is not `equals()`.
> 
> TODO(wfarner): Add a test case outside the scope of storage code to point 
> specifically at the impedance mismatch that was addressed here.
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/tools/java/thrift_wrapper_codegen.py 
> b5f2bc9e54b525a6a782d8873c9112f6496cd3f2 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> 295974a9f97e020dce11474d500a1bcd40d9f5d5 
> 
> Diff: https://reviews.apache.org/r/38079/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

Re: Review Request 37818: Moved executor settings configuration to loadable JSON

2015-09-02 Thread Renan DelValle 


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 17-18
> > 
> >
> > Can you omit this since it's blank?

Yep, was just leaving it there for now so I don't forget to put it in the 
documaenttion for this feature.


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > 
> >
> > The code later converts this array into a single command string.  I 
> > suggest we just make this a string here (and save some code down the line).

We can go ahead and do that. Kevin thought it should be the array way, but I'm 
not sure how strongly he feels about it. The extra code isn't too bad because 
we can use String.join() form Java 8. Either way, I'm fine with either. The 
single string may eliminate any suspicions that we're breaking something in a 
command when joining the array.


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, line 32
> > 
> >
> > Use single quotes to avoid escaping:
> > 
> > "'Hello World from Aurora!'"

Done


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java, lines 
> > 84-115
> > 
> >
> > Please add an entry in /NEWS under 0.10.0 to inform people what args 
> > were removed, and what the replacement is.

Done.


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, lines 71-73
> > 
> >
> > remove

Done


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, lines 
> > 107-108
> > 
> >
> > I suggest removing this for now.  There's a lot of ways fetching the 
> > URIs and executing commands can fail.  For the sake of simplicity, let's 
> > allow them to fail and focus on providing good visibility into the failure.

Done


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java,
> >  line 46
> > 
> >
> > please add a javadoc for this class

Done


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java,
> >  line 62
> > 
> >
> > Save a few lines:
> > 
> > ```
> > String config = Files.toString(configFile, StandardCharsets.UTF_8);
> > ```

Awesome. Done


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java,
> >  line 79
> > 
> >
> > remove the ```\n```, same with the string below

Done and Done


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java, line 210
> > 
> >
> > throw Throwables.propagate(e);
> > 
> > Otherwise the app will exit in a non-obvious way.

Thanks for pointing it out. I meant to come back to it but ended up forgetting 
about it.


> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java,
> >  line 87
> > 
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on 
> > ExecutorSettings)
> > - avoid hand-rolling deserializers
> > 
> > This sounds like more work, but it's not that bad.  The major 
> > difference is that you'll create a class in this file that defines the JSON 
> > structure, and does so in a way that the json deserializer can work with.
> > 
> > Maxim just did something very similar, which you can crib from:
> > 
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > 
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
> 
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to 
> translate

Re: Review Request 38080: Remove openjdk runtime requirement.

2015-09-02 Thread Kevin Sweeney 


> On Sept. 2, 2015, 5:08 p.m., Bill Farner wrote:
> > Ship It!

procedural question - I see there's a long-running 0.9.x branch - should I 
merge this to master then merge this into that branch?


- Kevin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38080/#review97573
---


On Sept. 2, 2015, 4:36 p.m., Kevin Sweeney wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38080/
> ---
> 
> (Updated Sept. 2, 2015, 4:36 p.m.)
> 
> 
> Review request for Aurora, Steve Salevan and Bill Farner.
> 
> 
> Repository: aurora-packaging
> 
> 
> Description
> ---
> 
> Remove openjdk runtime requirement.
> 
> This addresses one of my +0s.
> 
> 
> Diffs
> -
> 
>   specs/rpm/aurora.spec a559b42d73a4211891d8c7569ea7ab2a3a6993ae 
> 
> Diff: https://reviews.apache.org/r/38080/diff/
> 
> 
> Testing
> ---
> 
> Using a clean vagrant environment installed aurora, then installed the oracle 
> jdk, then uninstalled java-1.8.0-openjdk-headless (that aurora brought in 
> transitively). RPM didn't complain and when I restarted the scheduler it was 
> running under the new JVM.
> 
> ```
> [vagrant@localhost ~]$ curl -s http://localhost:8081/vars | grep 
> 'jvm_prop_java_vendor '
> jvm_prop_java_vendor Oracle Corporation
> [vagrant@localhost ~]$ 
> ```
> 
> 
> Thanks,
> 
> Kevin Sweeney
> 
>

Re: Review Request 37818: Moved executor settings configuration to loadable JSON

2015-09-02 Thread Kevin Sweeney 


> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > 
> >
> > The code later converts this array into a single command string.  I 
> > suggest we just make this a string here (and save some code down the line).
> 
> Renan DelValle wrote:
> We can go ahead and do that. Kevin thought it should be the array way, 
> but I'm not sure how strongly he feels about it. The extra code isn't too bad 
> because we can use String.join() form Java 8. Either way, I'm fine with 
> either. The single string may eliminate any suspicions that we're breaking 
> something in a command when joining the array.

Bill, I prefer the array mode as it's more explicit for dealing with magic 
characters in shell commands (like " (){}$").

You can still explicitly do

```
["sh", "-c", "PATH=/opt/aurora/executor/bin:$PATH thermos"]
```

When you prefer the old form, but this form will not automatically expand 
environment variables or do other shell magic for you.

Renan - can you update CommandUtil to use the `arguments`[1] field instead of 
`shell` and `value` (or drop a TODO for a later patch)? Changing back to 
`String.join` defeats part of the purpose of the array form here.

[1] 
https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L408-L423


- Kevin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
---


On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> ---
> 
> (Updated Sept. 1, 2015, 7:35 p.m.)
> 
> 
> Review request for Aurora.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is the first stage in a series of patches to create support for custom 
> executors. In an effort to expedite the review process, I have decided to 
> break down my patch into multiple pieces that when/if commited won't break 
> the trunk.
> 
> This patch includes the ability to load configuration from a JSON file. A 
> JSON example file is included in examples/vagrant/executors-config.json
> 
> Command line arguments have been eliminated and moved over to the JSON file. 
> GSON is leveraged and does most of the work with the aid of a few custom 
> deserializers that were needed. 
> 
> Note that right now a global container mount that does not follow 
> specification will cause the scheduler to detect the error an exit early. It 
> is up for discussion if this is the desired behavior or if we should just 
> ignore said mount.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/executors-config.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
> 4f43892723db4744db205ea7dd107e9e9ce9d5db 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 8047622e206c9827e5cd8e40152a278d495bd0ff 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>  PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> 1a25924d789295c5950947f5e302e1d1fbec68f2 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java
>  PRE-CREATION 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json
>  PRE-CREATION 
>   
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json
>  PRE-CREATION 
>   
>

Re: Review Request 37818: Moved executor settings configuration to loadable JSON

2015-09-02 Thread Bill Farner 


> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > 
> >
> > The code later converts this array into a single command string.  I 
> > suggest we just make this a string here (and save some code down the line).
> 
> Renan DelValle wrote:
> We can go ahead and do that. Kevin thought it should be the array way, 
> but I'm not sure how strongly he feels about it. The extra code isn't too bad 
> because we can use String.join() form Java 8. Either way, I'm fine with 
> either. The single string may eliminate any suspicions that we're breaking 
> something in a command when joining the array.
> 
> Kevin Sweeney wrote:
> Bill, I prefer the array mode as it's more explicit for dealing with 
> magic characters in shell commands (like " (){}$").
> 
> You can still explicitly do
> 
> ```
> ["sh", "-c", "PATH=/opt/aurora/executor/bin:$PATH thermos"]
> ```
> 
> When you prefer the old form, but this form will not automatically expand 
> environment variables or do other shell magic for you.
> 
> Renan - can you update CommandUtil to use the `arguments`[1] field 
> instead of `shell` and `value` (or drop a TODO for a later patch)? Changing 
> back to `String.join` defeats part of the purpose of the array form here.
> 
> [1] 
> https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L408-L423

Aha - the `arguments` field in mesos.proto was the missing detail for me.  
Since mesos allows us to do the right thing here, we should definitely use an 
array.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
---


On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> ---
> 
> (Updated Sept. 1, 2015, 7:35 p.m.)
> 
> 
> Review request for Aurora.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is the first stage in a series of patches to create support for custom 
> executors. In an effort to expedite the review process, I have decided to 
> break down my patch into multiple pieces that when/if commited won't break 
> the trunk.
> 
> This patch includes the ability to load configuration from a JSON file. A 
> JSON example file is included in examples/vagrant/executors-config.json
> 
> Command line arguments have been eliminated and moved over to the JSON file. 
> GSON is leveraged and does most of the work with the aid of a few custom 
> deserializers that were needed. 
> 
> Note that right now a global container mount that does not follow 
> specification will cause the scheduler to detect the error an exit early. It 
> is up for discussion if this is the desired behavior or if we should just 
> ignore said mount.
> 
> 
> Diffs
> -
> 
>   examples/vagrant/executors-config.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
> 4f43892723db4744db205ea7dd107e9e9ce9d5db 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> e909451892f117e9e6eb80994079661827a0914c 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 8047622e206c9827e5cd8e40152a278d495bd0ff 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>  PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> 1a25924d789295c5950947f5e302e1d1fbec68f2 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java
>  PRE-CREATION 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
>

Re: Review Request 38081: Dropping quota check for dedicated jobs and exposing dedicated consumption.

2015-09-02 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38081/#review97590
---

Ship it!


Master (a459527) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Sept. 3, 2015, 12:46 a.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38081/
> ---
> 
> (Updated Sept. 3, 2015, 12:46 a.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Bill Farner, and Zameer Manji.
> 
> 
> Bugs: AURORA-1460
> https://issues.apache.org/jira/browse/AURORA-1460
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This change drops quota enforcement for dedicated jobs and adds dedicated 
> resource consumption into the thrift API. Feel free to pushback on the latter 
> but I think users will find it immediately confusing when we change the "Prod 
> consumption" to something like "Prod shared pool consumption" in the 
> UI/client and not provide the missing dedicated rollup numbers.
> 
> 
> Diffs
> -
> 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> 21137bbbdc3010c6b1e2cc0ebb3b99bfa8490563 
>   src/main/java/org/apache/aurora/scheduler/quota/QuotaInfo.java 
> b40ecd0af7c1d1bb9372bd89c741622ce4c9040c 
>   src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 
> 8787aeaa6655cfab1e0a6d5719f9e08a89df7631 
>   src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
> 3d89e43659750de63d7588f8574e7a350caea04b 
>   src/main/python/apache/aurora/client/api/quota_check.py 
> 75406ac1d3ec9ba655daf7c6125f208f74290cfb 
>   src/main/python/apache/aurora/client/cli/quota.py 
> e8aa010f5ecce0464a8ad6b072bccba589fe16d7 
>   src/main/resources/scheduler/assets/js/controllers.js 
> 98920196db34f2eb4dcad93429274517e7383efe 
>   src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 
> db60cd21d06d636505202bac7277a13dc24d46e6 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java
>  4d4e752088f7dca99675cc66782ae046bbd516d6 
>   src/test/python/apache/aurora/admin/test_admin.py 
> d793293acc7f77c0081968334c38b984d865fbc8 
>   src/test/python/apache/aurora/client/api/test_quota_check.py 
> 6c9bc373d1868bd6e0cae2f2218261f39fd7ab8f 
>   src/test/python/apache/aurora/client/cli/test_quota.py 
> 3573e4c4575577e2232d2b99ca781b06a03d48d7 
> 
> Diff: https://reviews.apache.org/r/38081/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 38084: Builder should run createrepo

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38084/#review97595
---

Ship it!


Thanks!!

- Bill Farner


On Sept. 2, 2015, 6:01 p.m., Kevin Sweeney wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38084/
> ---
> 
> (Updated Sept. 2, 2015, 6:01 p.m.)
> 
> 
> Review request for Aurora and Bill Farner.
> 
> 
> Repository: aurora-packaging
> 
> 
> Description
> ---
> 
> Builder should run createrepo
> 
> 
> Diffs
> -
> 
>   builder/rpm/centos-7/build.sh 912f7c788be648f8f3a11ec868a56d2994c62117 
> 
> Diff: https://reviews.apache.org/r/38084/diff/
> 
> 
> Testing
> ---
> 
> Tested on 0.9.x branch
> 
> 
> Thanks,
> 
> Kevin Sweeney
> 
>

Review Request 38086: Add test environments for RPMs and debs.

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38086/
---

Review request for Aurora and Kevin Sweeney.


Repository: aurora-packaging


Description
---

Add test environments for RPMs and debs.


Diffs
-

  .gitignore 849ddff3b7ec917b5f4563e9a6d3ea63ea512a70 
  test/deb/ubuntu-trusty/Vagrantfile PRE-CREATION 
  test/deb/ubuntu-trusty/provision.sh PRE-CREATION 
  test/rpm/centos-7/README.md PRE-CREATION 
  test/rpm/centos-7/Vagrantfile PRE-CREATION 
  test/rpm/centos-7/provision.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/38086/diff/


Testing
---


Thanks,

Bill Farner

Re: Review Request 38086: Add test environments for RPMs and debs.

2015-09-02 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38086/
---

(Updated Sept. 2, 2015, 7:59 p.m.)


Review request for Aurora and Kevin Sweeney.


Changes
---

Neglected to git-add ubuntu readme.


Repository: aurora-packaging


Description
---

Add test environments for RPMs and debs.


Diffs (updated)
-

  .gitignore 849ddff3b7ec917b5f4563e9a6d3ea63ea512a70 
  test/deb/ubuntu-trusty/README.md PRE-CREATION 
  test/deb/ubuntu-trusty/Vagrantfile PRE-CREATION 
  test/deb/ubuntu-trusty/provision.sh PRE-CREATION 
  test/rpm/centos-7/README.md PRE-CREATION 
  test/rpm/centos-7/Vagrantfile PRE-CREATION 
  test/rpm/centos-7/provision.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/38086/diff/


Testing
---


Thanks,

Bill Farner

Re: Review Request 37926: Adding oversubscription e2e tests.

2015-09-02 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37926/
---

(Updated Sept. 3, 2015, 4:23 a.m.)


Review request for Aurora, Bill Farner and Zameer Manji.


Changes
---

Adding ticket.


Bugs: AURORA-1464
https://issues.apache.org/jira/browse/AURORA-1464


Repository: aurora


Description
---

Also, had to backout client tier validation added earlier until the feature 
exits beta.


Diffs
-

  examples/vagrant/upstart/aurora-scheduler.conf 
e909451892f117e9e6eb80994079661827a0914c 
  examples/vagrant/upstart/mesos-slave.conf 
2b6a60673fc0a7ea3b73471701cd5d3efd6ce639 
  src/main/java/org/apache/aurora/scheduler/http/Offers.java 
4329ce10189df41a7a45cd6cc369333d7d5f65b0 
  src/main/python/apache/aurora/client/config.py 
f5ac084ae5b448221363add9476334742b217b92 
  src/test/python/apache/aurora/client/test_config.py 
37459f5218e76fac831601806a0331927c0ee2c0 
  src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 
c1a10d8ea60be6aa56e4517fb34288d7d5ae1480 
  src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 
870b3e68035fdf86253cf9b92b606645134b3369 
  src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora 
e55aad3a58d4e3c19332e06b70771f51f07aa9b7 
  src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 
423dd4d4e8b03c2f852e25acd9340bd6288b7d24 
  src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
018efbe43de0c829799bc2f789958c55b1cb61a4 

Diff: https://reviews.apache.org/r/37926/diff/


Testing
---


Thanks,

Maxim Khutornenko

Re: Review Request 37956: Only process instances if there's been a change.

2015-09-02 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37956/#review97503
---


Master (64c00f1) is red with this patch.
  ./build-support/jenkins/build.sh

page = self.process_index(url, page)

  File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 
405, in process_index

self.scan_url(new_url)

  File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 
769, in scan_url

self.process_url(url, True)

  File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 
313, in process_url

page = f.read()

  File "/usr/lib/python2.7/socket.py", line 351, in read

data = self._sock.recv(rbufsize)

  File "/usr/lib/python2.7/httplib.py", line 543, in read

return self._read_chunked(amt)

  File "/usr/lib/python2.7/httplib.py", line 603, in _read_chunked

value.append(self._safe_read(amt))

  File "/usr/lib/python2.7/httplib.py", line 658, in _safe_read

chunk = self.fp.read(min(amt, MAXAMOUNT))

  File "/usr/lib/python2.7/socket.py", line 380, in read

data = self._sock.recv(left)

socket.timeout: timed out


Cleaning up...
Command python setup.py egg_info failed with error code 1 in 
/tmp/user/2/pip_build_jenkins/protobuf
Storing debug log for failure in /home/jenkins/.pip/pip.log


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Sept. 2, 2015, 6:09 p.m., Joshua Cohen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37956/
> ---
> 
> (Updated Sept. 2, 2015, 6:09 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Bill Farner.
> 
> 
> Bugs: AURORA-1345
> https://issues.apache.org/jira/browse/AURORA-1345
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> The `true` parameter to `scope.$watch` tells Angular to only execute the 
> supplied function if the watched data has changed. Given our 15s polling 
> interval is likely less than time it takes for any progress to be made on an 
> update, this removes the majority of (no-op) DOM operations.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/js/directives.js 
> c02c26f51ec4f190e61a1e79e4e045e9765bc2a5 
> 
> Diff: https://reviews.apache.org/r/37956/diff/
> 
> 
> Testing
> ---
> 
> Left tab open continually polling a job with ~3000 instances, confirmed that 
> memory usage did not grow beyond ~200MB.
> 
> 
> Thanks,
> 
> Joshua Cohen
> 
>

Re: Review Request 38072: Revert "Refer to shared task_configs table for job updates."

2015-09-02 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38072/#review97543
---

Ship it!


Ship It!

- Maxim Khutornenko


On Sept. 2, 2015, 10:07 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38072/
> ---
> 
> (Updated Sept. 2, 2015, 10:07 p.m.)
> 
> 
> Review request for Aurora and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-1476
> https://issues.apache.org/jira/browse/AURORA-1476
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This patch switched the update store over to use part of the experimental 
> task store, which is affected by AURORA-1476.  Reverting until that issue is 
> fixed.
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 
> 493f75dbf6b9403e5fb5bb48306336880d54 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStore.java 
> 765213269c3014866ed8a5deafea9b6f667e1500 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java
>  a3b04949f1726f110638e4f93ef45947cdb9e7f8 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TaskConfigTypeHandler.java
>  PRE-CREATION 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java
>  ed561c65947d41861d80229ad459392a4bceadf1 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbInstanceTaskConfig.java
>  f3fd7a96326408a990ab5014b86336759902f462 
>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdate.java 
> 78703e92c932cd5e93ff0b70f2a0b6928f6b4003 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateDetails.java
>  3a5272471e15177a35397acea920198499d16f0b 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateInstructions.java
>  d19aa859bcc5da2c9c7d4ebbb8db307a1ea2883e 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbStoredJobUpdateDetails.java
>  8ec6d4710964fd64d24774a719baffa27502c607 
>   
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml
>  69e11b69c4d8e0e2cba4c30ecc56077b2a3245ad 
>   src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 
> 4df685a31b23ff40f42bb3a9b0278733cd722fd3 
>   
> src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java
>  4c7133fa66bbe14002e2dc22cfaefa1585ab8a22 
> 
> Diff: https://reviews.apache.org/r/38072/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

Re: Review Request 38072: Revert "Refer to shared task_configs table for job updates."

2015-09-02 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38072/#review97547
---

Ship it!


Master (85f93a7) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Sept. 2, 2015, 10:07 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38072/
> ---
> 
> (Updated Sept. 2, 2015, 10:07 p.m.)
> 
> 
> Review request for Aurora and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-1476
> https://issues.apache.org/jira/browse/AURORA-1476
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This patch switched the update store over to use part of the experimental 
> task store, which is affected by AURORA-1476.  Reverting until that issue is 
> fixed.
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 
> 493f75dbf6b9403e5fb5bb48306336880d54 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStore.java 
> 765213269c3014866ed8a5deafea9b6f667e1500 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java
>  a3b04949f1726f110638e4f93ef45947cdb9e7f8 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TaskConfigTypeHandler.java
>  PRE-CREATION 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java
>  ed561c65947d41861d80229ad459392a4bceadf1 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbInstanceTaskConfig.java
>  f3fd7a96326408a990ab5014b86336759902f462 
>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdate.java 
> 78703e92c932cd5e93ff0b70f2a0b6928f6b4003 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateDetails.java
>  3a5272471e15177a35397acea920198499d16f0b 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateInstructions.java
>  d19aa859bcc5da2c9c7d4ebbb8db307a1ea2883e 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbStoredJobUpdateDetails.java
>  8ec6d4710964fd64d24774a719baffa27502c607 
>   
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml
>  69e11b69c4d8e0e2cba4c30ecc56077b2a3245ad 
>   src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 
> 4df685a31b23ff40f42bb3a9b0278733cd722fd3 
>   
> src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java
>  4c7133fa66bbe14002e2dc22cfaefa1585ab8a22 
> 
> Diff: https://reviews.apache.org/r/38072/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Bill Farner
> 
>