Re: Review Request 40149: Create ExecutorSettings creation closer to command line arguments.

2015-11-11 Thread Bill Farner 


> On Nov. 10, 2015, 2:07 p.m., Zameer Manji wrote:
> > src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java, line 227
> > 
> >
> > Is there a reason why we are not using Java's URI type to do this sort 
> > of parsing?

First - i'd like to point out that this is not new code (it came from the 
now-deleted `CommandUtil`).  As for using URI, i'd like to play agnostic in 
this patch and not impose new format rules by using `URI` parsing.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40149/#review105966
---


On Nov. 10, 2015, 12:42 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40149/
> ---
> 
> (Updated Nov. 10, 2015, 12:42 p.m.)
> 
> 
> Review request for Aurora and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is prep work for https://issues.apache.org/jira/browse/AURORA-1376
> 
> The high-level goal is to push ExecutorInfo creation closer to command line 
> args so they can be fully specified there in subsequent changes.  A lot of 
> the changes here are code shuffle, but there is one behavior change in 
> `MesosTaskFactory`.  There is currently a dance where `RESOURCES_EPSILON` is 
> used to effectively account all resources under the task.  With this change, 
> the executor resources are used verbatim and kept distinct from the task's 
> resources.
> 
> 
> Diffs
> -
> 
>   config/legacy_untested_classes.txt b77265ca1c89f9a2d7e6e01dea9f04b200e4ef6b 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 
> c665c3237a03f0fadf8ca158b97cad14dd1eea81 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 99b874459c358c67234c6fe748e6427f313a04a8 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> fb0f7ccf7f4dfb3ccb2a4cb4a5643c4b5487ff8f 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorConfig.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/Executors.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> PRE-CREATION 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  67d7f072de045d4211b6239119802473278d6419 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> f63d6f12d114c062f81137ffc0b1078837e3cf76 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java
>  0576704f6bcdc2a7568ee1597613f71f33e092d9 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  8ce8da88fe705bfb9846a790f76bfeb6b4af2a06 
> 
> Diff: https://reviews.apache.org/r/40149/diff/
> 
> 
> Testing
> ---
> 
> End-to-end tests pass.
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

Re: Review Request 40149: Create ExecutorSettings creation closer to command line arguments.

2015-11-10 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40149/#review105956
---

Ship it!



src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java (line 219)


Any reason you couldn't use ResourceSlot.makeMesosResource instead?


- Maxim Khutornenko


On Nov. 10, 2015, 8:42 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40149/
> ---
> 
> (Updated Nov. 10, 2015, 8:42 p.m.)
> 
> 
> Review request for Aurora and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is prep work for https://issues.apache.org/jira/browse/AURORA-1376
> 
> The high-level goal is to push ExecutorInfo creation closer to command line 
> args so they can be fully specified there in subsequent changes.  A lot of 
> the changes here are code shuffle, but there is one behavior change in 
> `MesosTaskFactory`.  There is currently a dance where `RESOURCES_EPSILON` is 
> used to effectively account all resources under the task.  With this change, 
> the executor resources are used verbatim and kept distinct from the task's 
> resources.
> 
> 
> Diffs
> -
> 
>   config/legacy_untested_classes.txt b77265ca1c89f9a2d7e6e01dea9f04b200e4ef6b 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 
> c665c3237a03f0fadf8ca158b97cad14dd1eea81 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 99b874459c358c67234c6fe748e6427f313a04a8 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> fb0f7ccf7f4dfb3ccb2a4cb4a5643c4b5487ff8f 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorConfig.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/Executors.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> PRE-CREATION 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  67d7f072de045d4211b6239119802473278d6419 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> f63d6f12d114c062f81137ffc0b1078837e3cf76 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java
>  0576704f6bcdc2a7568ee1597613f71f33e092d9 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  8ce8da88fe705bfb9846a790f76bfeb6b4af2a06 
> 
> Diff: https://reviews.apache.org/r/40149/diff/
> 
> 
> Testing
> ---
> 
> End-to-end tests pass.
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

Re: Review Request 40149: Create ExecutorSettings creation closer to command line arguments.

2015-11-10 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40149/#review105957
---

Ship it!


Master (3ea1b26) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Nov. 10, 2015, 8:42 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40149/
> ---
> 
> (Updated Nov. 10, 2015, 8:42 p.m.)
> 
> 
> Review request for Aurora and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is prep work for https://issues.apache.org/jira/browse/AURORA-1376
> 
> The high-level goal is to push ExecutorInfo creation closer to command line 
> args so they can be fully specified there in subsequent changes.  A lot of 
> the changes here are code shuffle, but there is one behavior change in 
> `MesosTaskFactory`.  There is currently a dance where `RESOURCES_EPSILON` is 
> used to effectively account all resources under the task.  With this change, 
> the executor resources are used verbatim and kept distinct from the task's 
> resources.
> 
> 
> Diffs
> -
> 
>   config/legacy_untested_classes.txt b77265ca1c89f9a2d7e6e01dea9f04b200e4ef6b 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 
> c665c3237a03f0fadf8ca158b97cad14dd1eea81 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 99b874459c358c67234c6fe748e6427f313a04a8 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> fb0f7ccf7f4dfb3ccb2a4cb4a5643c4b5487ff8f 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorConfig.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/Executors.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> PRE-CREATION 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  67d7f072de045d4211b6239119802473278d6419 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> f63d6f12d114c062f81137ffc0b1078837e3cf76 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java
>  0576704f6bcdc2a7568ee1597613f71f33e092d9 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  8ce8da88fe705bfb9846a790f76bfeb6b4af2a06 
> 
> Diff: https://reviews.apache.org/r/40149/diff/
> 
> 
> Testing
> ---
> 
> End-to-end tests pass.
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

Review Request 40149: Create ExecutorSettings creation closer to command line arguments.

2015-11-10 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40149/
---

Review request for Aurora and Zameer Manji.


Repository: aurora


Description
---

This is prep work for https://issues.apache.org/jira/browse/AURORA-1376

The high-level goal is to push ExecutorInfo creation closer to command line 
args so they can be fully specified there in subsequent changes.  A lot of the 
changes here are code shuffle, but there is one behavior change in 
`MesosTaskFactory`.  There is currently a dance where `RESOURCES_EPSILON` is 
used to effectively account all resources under the task.  With this change, 
the executor resources are used verbatim and kept distinct from the task's 
resources.


Diffs
-

  config/legacy_untested_classes.txt b77265ca1c89f9a2d7e6e01dea9f04b200e4ef6b 
  src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
  src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
197184b6edc0768d677636341b5737f262abdf7d 
  src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 
c665c3237a03f0fadf8ca158b97cad14dd1eea81 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
99b874459c358c67234c6fe748e6427f313a04a8 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
fb0f7ccf7f4dfb3ccb2a4cb4a5643c4b5487ff8f 
  src/main/java/org/apache/aurora/scheduler/mesos/ExecutorConfig.java 
PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
b3c913892248e4a9a8111412307463985f5ca97f 
  src/main/java/org/apache/aurora/scheduler/mesos/Executors.java PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
  src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
PRE-CREATION 
  
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 
67d7f072de045d4211b6239119802473278d6419 
  src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
f63d6f12d114c062f81137ffc0b1078837e3cf76 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
cd0295780d41bc4e914583f195b37eaed28a46dc 
  
src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java
 0576704f6bcdc2a7568ee1597613f71f33e092d9 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
dddf7952d3f0e508cd736d5fb95e573267708d43 
  src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
d0987251b058988fcbfab16c1b138c37e0c5b8c6 
  
src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
 8ce8da88fe705bfb9846a790f76bfeb6b4af2a06 

Diff: https://reviews.apache.org/r/40149/diff/


Testing
---

End-to-end tests pass.


Thanks,

Bill Farner

Re: Review Request 40149: Create ExecutorSettings creation closer to command line arguments.

2015-11-10 Thread Zameer Manji 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40149/#review105966
---

Ship it!



src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java (line 227)


Is there a reason why we are not using Java's URI type to do this sort of 
parsing?


- Zameer Manji


On Nov. 10, 2015, 12:42 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40149/
> ---
> 
> (Updated Nov. 10, 2015, 12:42 p.m.)
> 
> 
> Review request for Aurora and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This is prep work for https://issues.apache.org/jira/browse/AURORA-1376
> 
> The high-level goal is to push ExecutorInfo creation closer to command line 
> args so they can be fully specified there in subsequent changes.  A lot of 
> the changes here are code shuffle, but there is one behavior change in 
> `MesosTaskFactory`.  There is currently a dance where `RESOURCES_EPSILON` is 
> used to effectively account all resources under the task.  With this change, 
> the executor resources are used verbatim and kept distinct from the task's 
> resources.
> 
> 
> Diffs
> -
> 
>   config/legacy_untested_classes.txt b77265ca1c89f9a2d7e6e01dea9f04b200e4ef6b 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> c210c0db07bb1f4b3f76668178dcd7e2de56a4ac 
>   src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 
> 197184b6edc0768d677636341b5737f262abdf7d 
>   src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 
> c665c3237a03f0fadf8ca158b97cad14dd1eea81 
>   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
> 99b874459c358c67234c6fe748e6427f313a04a8 
>   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
> aa5ce8b2f14c7dbd0eae120018ee41387c26059f 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> fb0f7ccf7f4dfb3ccb2a4cb4a5643c4b5487ff8f 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorConfig.java 
> PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 
> b3c913892248e4a9a8111412307463985f5ca97f 
>   src/main/java/org/apache/aurora/scheduler/mesos/Executors.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> f6ba2c40aea555d3e0ab774218bfe08d7e1c984b 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> PRE-CREATION 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  67d7f072de045d4211b6239119802473278d6419 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 6fad3344042dc6a75cdf74ce79d388fcd4fc9861 
>   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
> f63d6f12d114c062f81137ffc0b1078837e3cf76 
>   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
> cd0295780d41bc4e914583f195b37eaed28a46dc 
>   
> src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java
>  0576704f6bcdc2a7568ee1597613f71f33e092d9 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> dddf7952d3f0e508cd736d5fb95e573267708d43 
>   src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java 
> d0987251b058988fcbfab16c1b138c37e0c5b8c6 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  8ce8da88fe705bfb9846a790f76bfeb6b4af2a06 
> 
> Diff: https://reviews.apache.org/r/40149/diff/
> 
> 
> Testing
> ---
> 
> End-to-end tests pass.
> 
> 
> Thanks,
> 
> Bill Farner
> 
>