Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > Can you see any opportunity to break this diff apart?  As it stands i'm 
> > having a hard time giving a thoughtful review.  Perhaps you can start by 
> > introducing the `Assignment` class?
> 
> Maxim Khutornenko wrote:
> I'd really prefer keeping this diff as a whole. The Assignment class 
> would not make sense without the entire picture in mind.
> 
> Bill Farner wrote:
> It's okay if you commit them back-to-back, but we really need to exercise 
> a pattern for incrementally building features even if they appear as code 
> islands in the commit history.  Doing so makes it easier to review with 
> confidence, and more likely that you get faster review turnaround.

Splitting this RB into 4 parts for easier reviewing. Keeping it around for 
history though.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
---


On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Feb. 4, 2015, 11:38 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Real life testing in vagrant (see pictures) shows close to 50% improvement in 
> task scheduling performance.
> 
> Testing with JMH shows over 97% better perf when testing with disabled 
> preemptor (1 scheduling loop):
> ```
> Master
> Benchmark 
>  Mode  SamplesScore  ErrorUnits
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1008291046.074 ±   145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1007522269.050 ±   142446.265  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples   Score   Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100 204171.046 ±   3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 215854.129 ±   8959.851  ns/op
> ```
> 
> Testing with preemptor enabled and no tasks eligible for preemption gives 
> around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001767479.299 ±   26907.571  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1001538682.287 ±  119119.911  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001105731.141 ±   10040.721  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 939230.662 ±   11091.505  ns/op
> ```
> 
> Testing with preemptor enabled and running the worst case possible scenario 
> (every slave is eligible and all tasks are victims) yields the least 
> improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark 
>  Mode  Samples  ScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   11043701.243 ±  40550.259  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   10478631.055 ± 178833.158  ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
> avgt  100  116258653.000 ± 403080.017  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   10886116.889 ± 193934.324  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenc

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Bill Farner]

> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > Can you see any opportunity to break this diff apart?  As it stands i'm 
> > having a hard time giving a thoughtful review.  Perhaps you can start by 
> > introducing the `Assignment` class?
> 
> Maxim Khutornenko wrote:
> I'd really prefer keeping this diff as a whole. The Assignment class 
> would not make sense without the entire picture in mind.

It's okay if you commit them back-to-back, but we really need to exercise a 
pattern for incrementally building features even if they appear as code islands 
in the commit history.  Doing so makes it easier to review with confidence, and 
more likely that you get faster review turnaround.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
---


On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Feb. 4, 2015, 11:38 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Real life testing in vagrant (see pictures) shows close to 50% improvement in 
> task scheduling performance.
> 
> Testing with JMH shows over 97% better perf when testing with disabled 
> preemptor (1 scheduling loop):
> ```
> Master
> Benchmark 
>  Mode  SamplesScore  ErrorUnits
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1008291046.074 ±   145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1007522269.050 ±   142446.265  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples   Score   Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100 204171.046 ±   3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 215854.129 ±   8959.851  ns/op
> ```
> 
> Testing with preemptor enabled and no tasks eligible for preemption gives 
> around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001767479.299 ±   26907.571  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1001538682.287 ±  119119.911  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001105731.141 ±   10040.721  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 939230.662 ±   11091.505  ns/op
> ```
> 
> Testing with preemptor enabled and running the worst case possible scenario 
> (every slave is eligible and all tasks are victims) yields the least 
> improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark 
>  Mode  Samples  ScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   11043701.243 ±  40550.259  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   10478631.055 ± 178833.158  ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
> avgt  100  116258653.000 ± 403080.017  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   10886116.889 ± 193934.324  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   10182572.955 ±  35740.891  ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBen

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > Can you see any opportunity to break this diff apart?  As it stands i'm 
> > having a hard time giving a thoughtful review.  Perhaps you can start by 
> > introducing the `Assignment` class?

I'd really prefer keeping this diff as a whole. The Assignment class would not 
make sense without the entire picture in mind.


> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 210
> > 
> >
> > What motivates you to supply these as overridden methods rather than a 
> > configuration object?  The latter seems like better encapsulation.

Nothing in particular. It has grown organically this way. Perhaps it's time to 
refactor. I would like to address it separately though to avoid growing this 
diff any further. Will leave a TODO.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
---


On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Feb. 4, 2015, 11:38 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Real life testing in vagrant (see pictures) shows close to 50% improvement in 
> task scheduling performance.
> 
> Testing with JMH shows over 97% better perf when testing with disabled 
> preemptor (1 scheduling loop):
> ```
> Master
> Benchmark 
>  Mode  SamplesScore  ErrorUnits
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1008291046.074 ±   145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1007522269.050 ±   142446.265  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples   Score   Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100 204171.046 ±   3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 215854.129 ±   8959.851  ns/op
> ```
> 
> Testing with preemptor enabled and no tasks eligible for preemption gives 
> around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001767479.299 ±   26907.571  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1001538682.287 ±  119119.911  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001105731.141 ±   10040.721  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 939230.662 ±   11091.505  ns/op
> ```
> 
> Testing with preemptor enabled and running the worst case possible scenario 
> (every slave is eligible and all tasks are victims) yields the least 
> improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark 
>  Mode  Samples  ScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   11043701.243 ±  40550.259  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   10478631.055 ± 178833.158  ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
> avgt  100  116258653.000 ± 403080.017  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   108861

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Bill Farner]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
---


Can you see any opportunity to break this diff apart?  As it stands i'm having 
a hard time giving a thoughtful review.  Perhaps you can start by introducing 
the `Assignment` class?


src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java


What motivates you to supply these as overridden methods rather than a 
configuration object?  The latter seems like better encapsulation.


- Bill Farner


On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Feb. 4, 2015, 11:38 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Real life testing in vagrant (see pictures) shows close to 50% improvement in 
> task scheduling performance.
> 
> Testing with JMH shows over 97% better perf when testing with disabled 
> preemptor (1 scheduling loop):
> ```
> Master
> Benchmark 
>  Mode  SamplesScore  ErrorUnits
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1008291046.074 ±   145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1007522269.050 ±   142446.265  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples   Score   Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100 204171.046 ±   3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 215854.129 ±   8959.851  ns/op
> ```
> 
> Testing with preemptor enabled and no tasks eligible for preemption gives 
> around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001767479.299 ±   26907.571  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  1001538682.287 ±  119119.911  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples  Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  1001105731.141 ±   10040.721  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100 939230.662 ±   11091.505  ns/op
> ```
> 
> Testing with preemptor enabled and running the worst case possible scenario 
> (every slave is eligible and all tasks are victims) yields the least 
> improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark 
>  Mode  Samples  ScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   11043701.243 ±  40550.259  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   10478631.055 ± 178833.158  ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
> avgt  100  116258653.000 ± 403080.017  ns/op
> 
> This RB
> Benchmark 
>  Mode  Samples Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   10886116.889 ± 193934.324  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   10182572.955 ±  35740.891  ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
> avgt  100  113656994.000 ± 424163.759  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
> 8c11ef8bd6609f3e4d97ca154d922898f8362446 
>   src/jmh/java/org/apa

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71084
---


Master (b49e1a0) is red with this patch.
  ./build-support/jenkins/build.sh

 src.test.python.apache.aurora.client.cli.config
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.cron  
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.inspect   
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.job   
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.plugins   
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.quota 
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.sla   
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.supdate   
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.task  
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.update
 .   SUCCESS
 src.test.python.apache.aurora.client.cli.version   
 .   SUCCESS
 src.test.python.apache.aurora.client.config
 .   SUCCESS
 src.test.python.apache.aurora.client.hooks.hooked_api  
 .   SUCCESS
 src.test.python.apache.aurora.client.hooks.non_hooked_api  
 .   SUCCESS
 src.test.python.apache.aurora.common.test_aurora_job_key   
 .   SUCCESS
 src.test.python.apache.aurora.common.test_cluster  
 .   SUCCESS
 src.test.python.apache.aurora.common.test_cluster_option   
 .   SUCCESS
 src.test.python.apache.aurora.common.test_clusters 
 .   SUCCESS
 src.test.python.apache.aurora.common.test_http_signaler
 .   SUCCESS
 src.test.python.apache.aurora.common.test_pex_version  
 .   SUCCESS
 src.test.python.apache.aurora.common.test_shellify 
 .   SUCCESS
 src.test.python.apache.aurora.common.test_transport
 .   SUCCESS
 src.test.python.apache.aurora.config.test_base 
 .   SUCCESS
 
src.test.python.apache.aurora.config.test_constraint_parsing
.   SUCCESS
 src.test.python.apache.aurora.config.test_loader   
 .   SUCCESS
 src.test.python.apache.aurora.config.test_thrift   
 .   SUCCESS
 src.test.python.apache.aurora.executor.common.task_info
 .   SUCCESS
 src.test.python.apache.aurora.executor.executor_base   
 .   SUCCESS
 src.test.python.apache.aurora.executor.executor_detector   
 .   SUCCESS
 src.test.python.apache.aurora.executor.executor_vars   
 .   SUCCESS
 src.test.python.apache.aurora.executor.status_manager  
 .   SUCCESS
 src.test.python.apache.aurora.executor.thermos_task_runner 
 .   FAILURE
 src.test.python.apache.thermos.common.test_pathspec
 .   SUCCESS
 
src.test.python.apache.thermos.core.test_runner_integration 
.   SUCCESS
 src.test.python.apache.thermos.monitoring.test_disk
 .   SUCCESS
 
FAILURE


   FAILURE


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Feb. 4, 2015, 11:38 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
---

(Updated Feb. 4, 2015, 11:38 p.m.)


Review request for Aurora, Kevin Sweeney and Bill Farner.


Changes
---

Rebased and modified benchmarks to support victim-less preemption run.


Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909


Repository: aurora


Description (updated)
---

Modified the task offer/task matching logic to skip offer matching for tasks 
previously vetoed statically.

Real life testing in vagrant (see pictures) shows close to 50% improvement in 
task scheduling performance.

Testing with JMH shows over 97% better perf when testing with disabled 
preemptor (1 scheduling loop):
```
Master
Benchmark   
   Mode  SamplesScore  ErrorUnits
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  1008291046.074 ±   145251.995  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  1007522269.050 ±   142446.265  ns/op

This RB
Benchmark   
   Mode  Samples   Score   Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  100 204171.046 ±   3800.124  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  100 215854.129 ±   8959.851  ns/op
```

Testing with preemptor enabled and no tasks eligible for preemption gives 
around 40% improvement (2 scheduling loops):
```
Master
Benchmark   
   Mode  Samples  Score Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  1001767479.299 ±   26907.571  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  1001538682.287 ±  119119.911  ns/op

This RB
Benchmark   
   Mode  Samples  Score Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  1001105731.141 ±   10040.721  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  100 939230.662 ±   11091.505  ns/op
```

Testing with preemptor enabled and running the worst case possible scenario 
(every slave is eligible and all tasks are victims) yields the least 
improvement 2-3% (3 scheduling loops).
```
Master
Benchmark   
   Mode  Samples  ScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  100   11043701.243 ±  40550.259  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  100   10478631.055 ± 178833.158  ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
avgt  100  116258653.000 ± 403080.017  ns/op

This RB
Benchmark   
   Mode  Samples Score Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  100   10886116.889 ± 193934.324  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  100   10182572.955 ±  35740.891  ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark
avgt  100  113656994.000 ± 424163.759  ns/op
```


Diffs (updated)
-

  src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 
8c11ef8bd6609f3e4d97ca154d922898f8362446 
  src/jmh/java/org/apache/aurora/benchmark/Tasks.java 
1a35f9ee9e8e76def0f9bf5454cf8cbdf6a89c25 
  src/main/java/org/apache/aurora/scheduler/TaskVars.java 
f017cdd26ca40138a7e141f21613ed567314c399 
  src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
f66383830140e5eaba436f35ebb5192eee65947a 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
ce47ff152e303fd2116bc3b9e91c0c1a8f76f258 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
6a43bcd1719e8aa32fd3fcb7387d0318c3c0b804 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
f06fdaeb92e154d0982bdabed5df93e7bcba9048 
  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
e1c29747c9854cf75bf63f6f085cf40ca68989af 
  src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
4e7efb3c1214c3d193afd61f162713490e

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review66964
---

Ship it!


Master (8c49029) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Jan. 7, 2015, 12:26 a.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Jan. 7, 2015, 12:26 a.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf when testing 
> with disabled preemptor:
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> Testing with preemptor enabled and running the worst case possible scenario 
> (every slave is eligible and has task victims to evaluate) is still 16-22% 
> faster than the master.
> ```
> Master with cluster fillup 0.9:
> Benchmark 
> Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>   avgt  100  7170244.522 ± 230259.848  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
> avgt  100  8158184.909 ± 465853.379  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>  Mode  Samples Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   6050901.381 ±  182134.371  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   6425953.419 ±  163741.064  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFilterin

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
---

(Updated Jan. 7, 2015, 12:26 a.m.)


Review request for Aurora, Kevin Sweeney and Bill Farner.


Changes
---

Kevin's comments.


Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909


Repository: aurora


Description
---

Modified the task offer/task matching logic to skip offer matching for tasks 
previously vetoed statically.

Preliminary testing in vagrant (see pictures below) shows close to 50% 
improvement in task scheduling performance.

Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and 
https://reviews.apache.org/r/28731/) shows over 97% better perf when testing 
with disabled preemptor:
```
Master with cluster fillup 0.9:
Benchmark   
 Mode  SamplesScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  8291046.074 ± 145251.995  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  7522269.050 ± 142446.265  ns/op

This RB with cluster fillup 0.9:
Benchmark   
 Mode  Samples   Score  Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  204171.046 ± 3800.124  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  215854.129 ± 8959.851  ns/op
```

Testing with preemptor enabled and running the worst case possible scenario 
(every slave is eligible and has task victims to evaluate) is still 16-22% 
faster than the master.
```
Master with cluster fillup 0.9:
Benchmark   
  Mode  SamplesScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
  avgt  100  7170244.522 ± 230259.848  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
avgt  100  8158184.909 ± 465853.379  ns/op

This RB with cluster fillup 0.9:
Benchmark   
   Mode  Samples Score Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  100   6050901.381 ±  182134.371  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  100   6425953.419 ±  163741.064  ns/op
```


Diffs (updated)
-

  src/main/java/org/apache/aurora/scheduler/TaskVars.java 
f017cdd26ca40138a7e141f21613ed567314c399 
  src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
f66383830140e5eaba436f35ebb5192eee65947a 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
c2a342ce07bfb223193886038761f0da5230135d 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
1cb56f19c331508a1585077e9c4a98f52aac343b 
  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
e1c29747c9854cf75bf63f6f085cf40ca68989af 
  src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
4e7efb3c1214c3d193afd61f162713490eb8effb 
  src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
5647349854a5e04de749c4d809684a0066d4da06 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
6cc13231560996b144101eba36577f49017aba06 
  
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 
265c38d20136210e7639ac8ea915d307a4b72949 
  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
411a55a8d85f60bb2703468f2d69b64b2736eee4 

Diff: https://reviews.apache.org/r/28617/diff/


Testing
---

./gradlew -Pq build

Tested in vagrant


File Attachments


NoStaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png


Thanks,

Maxim Khutornenko

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

> On Dec. 24, 2014, 12:26 a.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/TaskVars.java, line 79
> > 
> >
> > How about either
> > 
> > 1) Make this a member function of VetoGroup (so it's guaranteed by the 
> > compiler to be implemented if we add a new group type).
> > 
> > 2) Make this a Function>.
> > 
> > 3) Make this a Function with a default for a missing 
> > value.
> > 
> > Right now we risk a runtime NullPointerException if we add a new 
> > VetoGroup implementation and forget to update this map in a different 
> > package.

#1 - metric name mapping does not logically fit into the VetoGroup
#2 - this would add extra heap churn (Optional creation) I am trying to avoid 
for this very high frequency call event handler
#3 - default string value would slip the build phase and manifest itself as an 
invalid metric name at runtime, which is arguably even worse.

How about a test case exhaustively covering all VetoGroup mappings instead? 
Added.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review65988
---


On Dec. 15, 2014, 11:12 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 15, 2014, 11:12 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf when testing 
> with disabled preemptor:
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> Testing with preemptor enabled and running the worst case possible scenario 
> (every slave is eligible and has task victims to evaluate) is still 16-22% 
> faster than the master.
> ```
> Master with cluster fillup 0.9:
> Benchmark 
> Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>   avgt  100  7170244.522 ± 230259.848  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
> avgt  100  8158184.909 ± 465853.379  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>  Mode  Samples Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   6050901.381 ±  182134.371  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   6425953.419 ±  163741.064  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Kevin Sweeney]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review65988
---



src/main/java/org/apache/aurora/scheduler/TaskVars.java


How about either

1) Make this a member function of VetoGroup (so it's guaranteed by the 
compiler to be implemented if we add a new group type).

2) Make this a Function>.

3) Make this a Function with a default for a missing 
value.

Right now we risk a runtime NullPointerException if we add a new VetoGroup 
implementation and forget to update this map in a different package.


- Kevin Sweeney


On Dec. 15, 2014, 3:12 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 15, 2014, 3:12 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf when testing 
> with disabled preemptor:
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> Testing with preemptor enabled and running the worst case possible scenario 
> (every slave is eligible and has task victims to evaluate) is still 16-22% 
> faster than the master.
> ```
> Master with cluster fillup 0.9:
> Benchmark 
> Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>   avgt  100  7170244.522 ± 230259.848  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
> avgt  100  8158184.909 ± 465853.379  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>  Mode  Samples Score Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
>avgt  100   6050901.381 ±  182134.371  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
>  avgt  100   6425953.419 ±  163741.064  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
---

(Updated Dec. 15, 2014, 11:12 p.m.)


Review request for Aurora, Kevin Sweeney and Bill Farner.


Changes
---

Updated test results.


Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909


Repository: aurora


Description (updated)
---

Modified the task offer/task matching logic to skip offer matching for tasks 
previously vetoed statically.

Preliminary testing in vagrant (see pictures below) shows close to 50% 
improvement in task scheduling performance.

Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and 
https://reviews.apache.org/r/28731/) shows over 97% better perf when testing 
with disabled preemptor:
```
Master with cluster fillup 0.9:
Benchmark   
 Mode  SamplesScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  8291046.074 ± 145251.995  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  7522269.050 ± 142446.265  ns/op

This RB with cluster fillup 0.9:
Benchmark   
 Mode  Samples   Score  Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  204171.046 ± 3800.124  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  215854.129 ± 8959.851  ns/op
```

Testing with preemptor enabled and running the worst case possible scenario 
(every slave is eligible and has task victims to evaluate) is still 16-22% 
faster than the master.
```
Master with cluster fillup 0.9:
Benchmark   
  Mode  SamplesScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
  avgt  100  7170244.522 ± 230259.848  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
avgt  100  8158184.909 ± 465853.379  ns/op

This RB with cluster fillup 0.9:
Benchmark   
   Mode  Samples Score Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark
   avgt  100   6050901.381 ±  182134.371  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark
 avgt  100   6425953.419 ±  163741.064  ns/op
```


Diffs
-

  src/main/java/org/apache/aurora/scheduler/TaskVars.java 
f017cdd26ca40138a7e141f21613ed567314c399 
  src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
f66383830140e5eaba436f35ebb5192eee65947a 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
c2a342ce07bfb223193886038761f0da5230135d 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
1cb56f19c331508a1585077e9c4a98f52aac343b 
  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
e1c29747c9854cf75bf63f6f085cf40ca68989af 
  src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
4e7efb3c1214c3d193afd61f162713490eb8effb 
  src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
5647349854a5e04de749c4d809684a0066d4da06 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
6cc13231560996b144101eba36577f49017aba06 
  
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 
265c38d20136210e7639ac8ea915d307a4b72949 
  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
411a55a8d85f60bb2703468f2d69b64b2736eee4 

Diff: https://reviews.apache.org/r/28617/diff/


Testing
---

./gradlew -Pq build

Tested in vagrant


File Attachments


NoStaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png


Thanks,

Maxim Khutornenko

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

> On Dec. 12, 2014, 10:28 a.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java, line 295
> > 
> >
> > staticallyBannedOffers is only filled but never cleared. Looks like we 
> > will be leaking memory.

Thanks for catching this! Added tests to validate cleanup.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64890
---


On Dec. 12, 2014, 6:05 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 12, 2014, 6:05 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64951
---

Ship it!


Master (cbe9c05) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Dec. 12, 2014, 6:05 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 12, 2014, 6:05 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
---

(Updated Dec. 12, 2014, 6:05 p.m.)


Review request for Aurora, Kevin Sweeney and Bill Farner.


Changes
---

CR comments.


Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909


Repository: aurora


Description
---

Modified the task offer/task matching logic to skip offer matching for tasks 
previously vetoed statically.

Preliminary testing in vagrant (see pictures below) shows close to 50% 
improvement in task scheduling performance.

Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and 
https://reviews.apache.org/r/28731/) shows over 97% better perf:

```
Master with cluster fillup 0.9:
Benchmark   
 Mode  SamplesScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  8291046.074 ± 145251.995  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  7522269.050 ± 142446.265  ns/op

This RB with cluster fillup 0.9:
Benchmark   
 Mode  Samples   Score  Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  204171.046 ± 3800.124  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  215854.129 ± 8959.851  ns/op
```


Diffs (updated)
-

  src/main/java/org/apache/aurora/scheduler/TaskVars.java 
f017cdd26ca40138a7e141f21613ed567314c399 
  src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
f66383830140e5eaba436f35ebb5192eee65947a 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
c2a342ce07bfb223193886038761f0da5230135d 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
1cb56f19c331508a1585077e9c4a98f52aac343b 
  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
e1c29747c9854cf75bf63f6f085cf40ca68989af 
  src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
4e7efb3c1214c3d193afd61f162713490eb8effb 
  src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
5647349854a5e04de749c4d809684a0066d4da06 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
6cc13231560996b144101eba36577f49017aba06 
  
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 
265c38d20136210e7639ac8ea915d307a4b72949 
  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
411a55a8d85f60bb2703468f2d69b64b2736eee4 

Diff: https://reviews.apache.org/r/28617/diff/


Testing
---

./gradlew -Pq build

Tested in vagrant


File Attachments


NoStaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png


Thanks,

Maxim Khutornenko

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Stephan Erb]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64890
---



src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java


staticallyBannedOffers is only filled but never cleared. Looks like we will 
be leaking memory.


- Stephan Erb


On Dec. 8, 2014, 10:57 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 8, 2014, 10:57 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> ead9d28100673440168a32d114ecaa15874978a6 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Bill Farner]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64609
---


Heads up - i'm focusing on https://reviews.apache.org/r/28731/ before this 
review.

- Bill Farner


On Dec. 8, 2014, 10:57 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 8, 2014, 10:57 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> ead9d28100673440168a32d114ecaa15874978a6 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64314
---

Ship it!


Master (a9e1217) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Dec. 8, 2014, 10:57 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 8, 2014, 10:57 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> ead9d28100673440168a32d114ecaa15874978a6 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
---

(Updated Dec. 8, 2014, 10:57 p.m.)


Review request for Aurora, Kevin Sweeney and Bill Farner.


Changes
---

CR comments.


Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909


Repository: aurora


Description
---

Modified the task offer/task matching logic to skip offer matching for tasks 
previously vetoed statically.

Preliminary testing in vagrant (see pictures below) shows close to 50% 
improvement in task scheduling performance.

Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and 
https://reviews.apache.org/r/28731/) shows over 97% better perf:

```
Master with cluster fillup 0.9:
Benchmark   
 Mode  SamplesScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  8291046.074 ± 145251.995  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  7522269.050 ± 142446.265  ns/op

This RB with cluster fillup 0.9:
Benchmark   
 Mode  Samples   Score  Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  204171.046 ± 3800.124  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  215854.129 ± 8959.851  ns/op
```


Diffs (updated)
-

  src/main/java/org/apache/aurora/scheduler/TaskVars.java 
f017cdd26ca40138a7e141f21613ed567314c399 
  src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
f66383830140e5eaba436f35ebb5192eee65947a 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
ead9d28100673440168a32d114ecaa15874978a6 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
c2a342ce07bfb223193886038761f0da5230135d 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
1cb56f19c331508a1585077e9c4a98f52aac343b 
  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
e1c29747c9854cf75bf63f6f085cf40ca68989af 
  src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
4e7efb3c1214c3d193afd61f162713490eb8effb 
  src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
5647349854a5e04de749c4d809684a0066d4da06 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
6cc13231560996b144101eba36577f49017aba06 
  
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 
265c38d20136210e7639ac8ea915d307a4b72949 
  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
411a55a8d85f60bb2703468f2d69b64b2736eee4 

Diff: https://reviews.apache.org/r/28617/diff/


Testing
---

./gradlew -Pq build

Tested in vagrant


File Attachments


NoStaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png


Thanks,

Maxim Khutornenko

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

> On Dec. 8, 2014, 7:29 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, 
> > line 102
> > 
> >
> > Can you add this as a member to the Veto interface instead? This will 
> > ensure we don't add new Veto types without updating this map.
> > 
> > Like
> > 
> > ```java
> > interface Veto {
> >   VetoGroup getVetoGroup();
> > }
> > ```

There is no Veto interface and I am hesitant to create one just for this 
method. I have moved the group assignment into the VetoType instead. That also 
helped to get rid of a null check code not reachable from unit tests.


> On Dec. 8, 2014, 7:29 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, 
> > line 181
> > 
> >
> > If you follow the comment above this block becomes unnecessary.

Gone.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64259
---


On Dec. 5, 2014, 10:57 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 5, 2014, 10:57 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> ead9d28100673440168a32d114ecaa15874978a6 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Than

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Kevin Sweeney]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64259
---



src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java


Can you add this as a member to the Veto interface instead? This will 
ensure we don't add new Veto types without updating this map.

Like

```java
interface Veto {
  VetoGroup getVetoGroup();
}
```



src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java


If you follow the comment above this block becomes unnecessary.


- Kevin Sweeney


On Dec. 5, 2014, 2:57 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 5, 2014, 2:57 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> ead9d28100673440168a32d114ecaa15874978a6 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64111
---

Ship it!


Master (40d2718) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Dec. 5, 2014, 10:57 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 5, 2014, 10:57 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> Preliminary testing in vagrant (see pictures below) shows close to 50% 
> improvement in task scheduling performance.
> 
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and 
> https://reviews.apache.org/r/28731/) shows over 97% better perf:
> 
> ```
> Master with cluster fillup 0.9:
> Benchmark 
>Mode  SamplesScoreError  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  8291046.074 ± 145251.995  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  7522269.050 ± 142446.265  ns/op
> 
> This RB with cluster fillup 0.9:
> Benchmark 
>Mode  Samples   Score  Error  Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example   
>avgt  100  204171.046 ± 3800.124  ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example 
>avgt  100  215854.129 ± 8959.851  ns/op
> ```
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> ead9d28100673440168a32d114ecaa15874978a6 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
---

(Updated Dec. 5, 2014, 10:57 p.m.)


Review request for Aurora, Kevin Sweeney and Bill Farner.


Changes
---

Improving banned offer lookup performance.


Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909


Repository: aurora


Description (updated)
---

Modified the task offer/task matching logic to skip offer matching for tasks 
previously vetoed statically.

Preliminary testing in vagrant (see pictures below) shows close to 50% 
improvement in task scheduling performance.

Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and 
https://reviews.apache.org/r/28731/) shows over 97% better perf:

```
Master with cluster fillup 0.9:
Benchmark   
 Mode  SamplesScoreError  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  8291046.074 ± 145251.995  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  7522269.050 ± 142446.265  ns/op

This RB with cluster fillup 0.9:
Benchmark   
 Mode  Samples   Score  Error  Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example 
 avgt  100  204171.046 ± 3800.124  ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example   
 avgt  100  215854.129 ± 8959.851  ns/op
```


Diffs (updated)
-

  src/main/java/org/apache/aurora/scheduler/TaskVars.java 
f017cdd26ca40138a7e141f21613ed567314c399 
  src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
f66383830140e5eaba436f35ebb5192eee65947a 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
ead9d28100673440168a32d114ecaa15874978a6 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
c2a342ce07bfb223193886038761f0da5230135d 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
1cb56f19c331508a1585077e9c4a98f52aac343b 
  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
e1c29747c9854cf75bf63f6f085cf40ca68989af 
  src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
4e7efb3c1214c3d193afd61f162713490eb8effb 
  src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
5647349854a5e04de749c4d809684a0066d4da06 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
6cc13231560996b144101eba36577f49017aba06 
  
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 
265c38d20136210e7639ac8ea915d307a4b72949 
  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
411a55a8d85f60bb2703468f2d69b64b2736eee4 

Diff: https://reviews.apache.org/r/28617/diff/


Testing
---

./gradlew -Pq build

Tested in vagrant


File Attachments


NoStaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png


Thanks,

Maxim Khutornenko

Re: Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Aurora ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review63627
---

Ship it!


Master (552ae93) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On Dec. 3, 2014, 1:15 a.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> ---
> 
> (Updated Dec. 3, 2014, 1:15 a.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney and Bill Farner.
> 
> 
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Modified the task offer/task matching logic to skip offer matching for tasks 
> previously vetoed statically.
> 
> I will follow up with more accurate measurements 
> (https://reviews.apache.org/r/28474/) but preliminary testing in vagrant (see 
> pictures below) shows close to 50% improvement in task scheduling performance.
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/TaskVars.java 
> f017cdd26ca40138a7e141f21613ed567314c399 
>   src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
> f66383830140e5eaba436f35ebb5192eee65947a 
>   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
> ead9d28100673440168a32d114ecaa15874978a6 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
> c2a342ce07bfb223193886038761f0da5230135d 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
> 1cb56f19c331508a1585077e9c4a98f52aac343b 
>   src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
> e1c29747c9854cf75bf63f6f085cf40ca68989af 
>   src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
> 4e7efb3c1214c3d193afd61f162713490eb8effb 
>   src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
> 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
> 5647349854a5e04de749c4d809684a0066d4da06 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
> 6cc13231560996b144101eba36577f49017aba06 
>   
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java
>  265c38d20136210e7639ac8ea915d307a4b72949 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> 411a55a8d85f60bb2703468f2d69b64b2736eee4 
> 
> Diff: https://reviews.apache.org/r/28617/diff/
> 
> 
> Testing
> ---
> 
> ./gradlew -Pq build
> 
> Tested in vagrant
> 
> 
> File Attachments
> 
> 
> NoStaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
>   
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Review Request 28617: Implemented offer filtering for tasks with static vetoes.

[Maxim Khutornenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
---

Review request for Aurora, Kevin Sweeney and Bill Farner.


Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909


Repository: aurora


Description
---

Modified the task offer/task matching logic to skip offer matching for tasks 
previously vetoed statically.

I will follow up with more accurate measurements 
(https://reviews.apache.org/r/28474/) but preliminary testing in vagrant (see 
pictures below) shows close to 50% improvement in task scheduling performance.


Diffs
-

  src/main/java/org/apache/aurora/scheduler/TaskVars.java 
f017cdd26ca40138a7e141f21613ed567314c399 
  src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java 
f66383830140e5eaba436f35ebb5192eee65947a 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
ead9d28100673440168a32d114ecaa15874978a6 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 
c2a342ce07bfb223193886038761f0da5230135d 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 
1cb56f19c331508a1585077e9c4a98f52aac343b 
  src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 
e1c29747c9854cf75bf63f6f085cf40ca68989af 
  src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 
4e7efb3c1214c3d193afd61f162713490eb8effb 
  src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 
4cf602ad32b972c18eb5a81e9b2f59c67859bdb2 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 
5647349854a5e04de749c4d809684a0066d4da06 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 
6cc13231560996b144101eba36577f49017aba06 
  
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 
265c38d20136210e7639ac8ea915d307a4b72949 
  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
411a55a8d85f60bb2703468f2d69b64b2736eee4 

Diff: https://reviews.apache.org/r/28617/diff/


Testing
---

./gradlew -Pq build

Tested in vagrant


File Attachments


NoStaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
  
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png


Thanks,

Maxim Khutornenko