date:20160328

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread George Sirois


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/
---

(Updated March 29, 2016, 5:49 a.m.)


Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.


Repository: aurora


Description
---

In instances where the root filesystem is read-only, it is desirable to
have the executor/runner extract themselves into the sandbox.


Diffs (updated)
-

  RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
  config/legacy_untested_classes.txt 144b258b7a7f73131f07826a0fdcac04834d87db 
  docs/operations/configuration.md f9e8844914b7af2b0057ca5f1d7d4391a63ca142 
  
src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
 949c299bdbc54f976db994266fb97f3099256f13 
  
src/test/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModuleTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/45396/diff/


Testing
---


Thanks,

George Sirois

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread Joshua Cohen



> On March 29, 2016, 12:03 a.m., Joshua Cohen wrote:
> > Are there any concerns about any other unintended side effect of setting 
> > $HOME to the sandbox? I suppose it's opt-in for now, so that allays most 
> > fears.
> 
> George Sirois wrote:
> I couldn't think of any (nor did I discover any in testing, both with and 
> without Docker), but making it opt-in should allow us to gain experience with 
> running it in production environments without impacting existing deployments. 
> The reason I used HOME (as opposed to PEX_ROOT) is that the executor strips 
> PEX_ROOT before forking the runner, which has the same issue. We could add 
> some plumbing to get around that, but it seems like if you're running with a 
> read-only root FS, you'd want HOME to be a writable location anyway.

Yeah, I'm not coming up with anything either other than potentially causing 
tasks to exceed their disk space allotment due to ~/.pex/install now being 
counted against them. That's arguably the right thing to do though, operators 
should just be cognizant of that when enabling this (or if/when we make it the 
default).


- Joshua


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/#review125779
---


On March 28, 2016, 6:21 p.m., George Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45396/
> ---
> 
> (Updated March 28, 2016, 6:21 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> In instances where the root filesystem is read-only, it is desirable to
> have the executor/runner extract themselves into the sandbox.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  949c299bdbc54f976db994266fb97f3099256f13 
> 
> Diff: https://reviews.apache.org/r/45396/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> George Sirois
> 
>

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread George Sirois



> On March 29, 2016, 12:03 a.m., Joshua Cohen wrote:
> > Are there any concerns about any other unintended side effect of setting 
> > $HOME to the sandbox? I suppose it's opt-in for now, so that allays most 
> > fears.

I couldn't think of any (nor did I discover any in testing, both with and 
without Docker), but making it opt-in should allow us to gain experience with 
running it in production environments without impacting existing deployments. 
The reason I used HOME (as opposed to PEX_ROOT) is that the executor strips 
PEX_ROOT before forking the runner, which has the same issue. We could add some 
plumbing to get around that, but it seems like if you're running with a 
read-only root FS, you'd want HOME to be a writable location anyway.


- George


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/#review125779
---


On March 28, 2016, 6:21 p.m., George Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45396/
> ---
> 
> (Updated March 28, 2016, 6:21 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> In instances where the root filesystem is read-only, it is desirable to
> have the executor/runner extract themselves into the sandbox.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  949c299bdbc54f976db994266fb97f3099256f13 
> 
> Diff: https://reviews.apache.org/r/45396/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> George Sirois
> 
>

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread George Sirois



> On March 29, 2016, 12:03 a.m., Joshua Cohen wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java,
> >  line 112
> > 
> >
> > Given that this is a static method, it should be pretty easy to test, 
> > would you mind adding a test case for this behavior (probably requires 
> > refactoring the method to take the arg values as parameters, rather than 
> > referencing them directly).

Sure, will do.


- George


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/#review125779
---


On March 28, 2016, 6:21 p.m., George Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45396/
> ---
> 
> (Updated March 28, 2016, 6:21 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> In instances where the root filesystem is read-only, it is desirable to
> have the executor/runner extract themselves into the sandbox.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  949c299bdbc54f976db994266fb97f3099256f13 
> 
> Diff: https://reviews.apache.org/r/45396/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> George Sirois
> 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Bill Farner



> On March 22, 2016, 7:23 p.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, line 84
> > 
> >
> > I think we should also require that at least one field is specified in 
> > `permissions`.  Seems like the ACL entry would be meaningliess otherwise.
> 
> Kunal Thakar wrote:
> Done. Will it be an overkill to use something like jsonschema to validate 
> this instead?
> 
> Bill Farner wrote:
> I'll let you assess that for the time being.  I would not be opposed.  
> You should also consider defining a schema in pystachio, however.

Here's the schema fleshed out in pystachio.  I think this is how you should 
proceed.

```python
from apache.thermos.config.schema import (
Boolean, 
Default,
List,
Required,
String,
Struct
)

class Auth(Struct):
  scheme = Required(String)
  credential = Required(String)


class Permissions(Struct):
  read= Default(Boolean, False)
  write   = Default(Boolean, False)
  create  = Default(Boolean, False)
  delete  = Default(Boolean, False)


class Access(Struct):
  scheme  = Required(String)
  credential  = Required(String)
  permissions = Required(Permissions)


class ZkAuth(Struct):
  auth = Default(List(Auth), [])
  acl  = Default(List(Access), [])
```

If you're not familiar with pystachio, i suggest you drop into a repl to get a 
feel for the API:

`./pants -q repl src/main/python/apache/aurora/client`

>From there, you can paste the above classes and try your example schema:
```
>>> example = '''{
...   "auth": [
... {
...   "scheme": "digest",
...   "credential": "user:pass"
... }
...   ],
...   "acl": [
... {
...   "scheme": "digest",
...   "credential": "user:smGaoVKd/cQkjm7b88GyorAUz20=",
...   "permissions": {
... "read": true,
... "write": true,
... "create": true,
... "delete": true
...   }
... }
...   ]
... }'''
>>> a = ZkAuth.json_loads(example, strict=True)
>>> print a
ZkAuth(auth=AuthList(Auth(credential=user:pass,
 scheme=digest)),
   acl=AccessList(Access(credential=user:smGaoVKd/cQkjm7b88GyorAUz20=,
   scheme=digest,
   permissions=Permissions(read=True,
write=True,
create=True,
delete=True
>>> for auth in a.auth():
...   print auth.credential()
...
user:pass
```


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review124935
---


On March 28, 2016, 4:10 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 28, 2016, 4:10 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> (Updated JSON format for config file)
> ```json
> {
>   "auth": [
> {
>   "scheme": "",
>   "credential": ""
> }
>   ],
>   "acls": [
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
>   ]
> }
> ```
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/operations/security.md 1a3d9b7e7ba4ec1952dc886d5fbeb6b85d994fb9 
>   examples/vagrant/announcer-auth.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py
>  e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
>   src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
> PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> e1c12bbd4382c31e576439f6693d82d5661029b9 
>   src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
> fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> /vagrant/src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Bill Farner


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review125781
---




examples/vagrant/announcer-auth.json (line 8)


nit: this really should be `acl` (it's a single Access Control List).


- Bill Farner


On March 28, 2016, 4:10 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 28, 2016, 4:10 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> (Updated JSON format for config file)
> ```json
> {
>   "auth": [
> {
>   "scheme": "",
>   "credential": ""
> }
>   ],
>   "acls": [
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
>   ]
> }
> ```
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/operations/security.md 1a3d9b7e7ba4ec1952dc886d5fbeb6b85d994fb9 
>   examples/vagrant/announcer-auth.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py
>  e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
>   src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
> PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> e1c12bbd4382c31e576439f6693d82d5661029b9 
>   src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
> fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> /vagrant/src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh
> /vagrant/src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread Joshua Cohen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/#review125779
---



Are there any concerns about any other unintended side effect of setting $HOME 
to the sandbox? I suppose it's opt-in for now, so that allays most fears.


src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
 (line 112)


Given that this is a static method, it should be pretty easy to test, would 
you mind adding a test case for this behavior (probably requires refactoring 
the method to take the arg values as parameters, rather than referencing them 
directly).


- Joshua Cohen


On March 28, 2016, 6:21 p.m., George Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45396/
> ---
> 
> (Updated March 28, 2016, 6:21 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> In instances where the root filesystem is read-only, it is desirable to
> have the executor/runner extract themselves into the sandbox.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  949c299bdbc54f976db994266fb97f3099256f13 
> 
> Diff: https://reviews.apache.org/r/45396/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> George Sirois
> 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Bill Farner


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review125778
---




RELEASE-NOTES.md (line 15)


Link is now dead - should be 
`docs/operations/security.md#announcer-authentication`


- Bill Farner


On March 28, 2016, 4:10 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 28, 2016, 4:10 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> (Updated JSON format for config file)
> ```json
> {
>   "auth": [
> {
>   "scheme": "",
>   "credential": ""
> }
>   ],
>   "acls": [
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
>   ]
> }
> ```
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/operations/security.md 1a3d9b7e7ba4ec1952dc886d5fbeb6b85d994fb9 
>   examples/vagrant/announcer-auth.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py
>  e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
>   src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
> PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> e1c12bbd4382c31e576439f6693d82d5661029b9 
>   src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
> fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> /vagrant/src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh
> /vagrant/src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Bill Farner

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review125773
---

Great to see that you got it working in end-to-end tests!  A few things left to 
clean up; we're very close.

examples/vagrant/announcer-auth.json (line 8)

Tying in with my comment towards the end - let's add 'world read' access 
here.  I believe that would be represented as:

```
{
  "scheme": "world",
  "credential": "anyone",
  "permissions": {
"read": true
  }
}
```

examples/vagrant/announcer-auth.json (line 11)

I now have to backpedal on my advice to store the encrypted credentials 
here.  Since our hand is forced to store plaintext for the auth section, we 
might as well make this part plaintext too.  That leaves us with the burden of 
handling the digest step, but that shouldn't be too bad.

src/main/python/apache/aurora/executor/common/announcer.py (line 125)

A dict isn't a great substitute for a class.  I'd rather see something like 
this, which contains all data/logic associated with the config parsing:

```python
class ZkAuthConfig(object):
  def __init__(self, auths, acls):
self.auths = auths
self.acls = acls

  @staticmethod
  def from_file(zk_auth_config):
# make_zk_auth body
```

src/main/python/apache/aurora/executor/common/announcer.py (line 145)

Echoing past comment on truthiness - you almost certainly want `if 
zk_auth_config is None`

src/main/python/apache/aurora/executor/common/announcer.py (line 230)

Avoid mutable kwarg defaults, they will leave you in a fun multi-hour debug 
session one day!

Use `zk_auth=None` instead

Background here:

http://stackoverflow.com/questions/1132941/least-astonishment-in-python-the-mutable-default-argument

src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh (line 1)

I apologize if i gave the impression that another end-to-end test routine 
was needed, that was not the intent!  I merely intended for the ZK announcer 
auth to be 'enabled' in the vagrant environment by default, all the time.  I 
assume that is just a matter of a few bits of configuration, and little if 
anything else.

Feel free to nuke this file; i hope you at least learned something along 
the way!

src/test/sh/org/apache/aurora/e2e/validate_serverset.py (lines 31 - 60)

How about we simplify this and grant world-read access in 
`announcer-auth.json`?  In addition to making the test easier, i think it 
represents a common real-world configuration.

- Bill Farner

On March 28, 2016, 4:10 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 28, 2016, 4:10 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> (Updated JSON format for config file)
> ```json
> {
>   "auth": [
> {
>   "scheme": "",
>   "credential": ""
> }
>   ],
>   "acls": [
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
>   ]
> }
> ```
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/operations/security.md 1a3d9b7e7ba4ec1952dc886d5fbeb6b85d994fb9 
>   examples/vagrant/announcer-auth.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py
>  e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review125775
---


Ship it!




Master (f28f41a) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 28, 2016, 11:10 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 28, 2016, 11:10 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> (Updated JSON format for config file)
> ```json
> {
>   "auth": [
> {
>   "scheme": "",
>   "credential": ""
> }
>   ],
>   "acls": [
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
>   ]
> }
> ```
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/operations/security.md 1a3d9b7e7ba4ec1952dc886d5fbeb6b85d994fb9 
>   examples/vagrant/announcer-auth.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py
>  e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
>   src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
> PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> e1c12bbd4382c31e576439f6693d82d5661029b9 
>   src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
> fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> /vagrant/src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh
> /vagrant/src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Kunal Thakar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/
---

(Updated March 28, 2016, 11:10 p.m.)


Review request for Aurora, Bill Farner and Zameer Manji.


Changes
---

rebase


Repository: aurora


Description (updated)
---

Add ACL support for announcer
https://issues.apache.org/jira/browse/AURORA-1643

Adding support for service discovery ZK authentication. ZK authentication 
secrets should be stored in a file as json (as follows):
(Updated JSON format for config file)
```json
{
  "auth": [
{
  "scheme": "",
  "credential": ""
}
  ],
  "acls": [
{
  "scheme": "",
  "credential": "",
  "permissions": {
"read": ,
"write": ,
"create": ,
"delete": ,
"admin": ,
"all": 
  }
}
  ]
}
```


Diffs (updated)
-

  RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
  docs/operations/security.md 1a3d9b7e7ba4ec1952dc886d5fbeb6b85d994fb9 
  examples/vagrant/announcer-auth.json PRE-CREATION 
  examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
6634506108c346f8c23b2da7cc8d20d09d07d590 
  src/main/python/apache/aurora/executor/common/announcer.py 
79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
  
src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 
e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
  src/test/python/apache/aurora/executor/common/test_announcer.py 
142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
  src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
e1c12bbd4382c31e576439f6693d82d5661029b9 
  src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 

Diff: https://reviews.apache.org/r/45042/diff/


Testing (updated)
---

/vagrant/src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh
/vagrant/src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh


Thanks,

Kunal Thakar

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Bill Farner



> On March 22, 2016, 7:23 p.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, line 84
> > 
> >
> > I think we should also require that at least one field is specified in 
> > `permissions`.  Seems like the ACL entry would be meaningliess otherwise.
> 
> Kunal Thakar wrote:
> Done. Will it be an overkill to use something like jsonschema to validate 
> this instead?

I'll let you assess that for the time being.  I would not be opposed.  You 
should also consider defining a schema in pystachio, however.


> On March 22, 2016, 7:23 p.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, lines 186-187
> > 
> >
> > Feeling some deja vu here - the double underscore is unconventional - 
> > please change to single underscore.
> 
> Kunal Thakar wrote:
> There are plenty of double underscores throughout the file that predate 
> my changes. I am following the single underscore for new code. Should I 
> refactor the entire file?

Yes please do, it's nice when source files are at least internally consistent.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review124935
---


On March 28, 2016, 3:51 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 28, 2016, 3:51 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> ```json
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
> ```
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 6e9364e34db6dbb270468db3ff333b956c6bf9f3 
>   docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
>   examples/vagrant/announcer-auth.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py
>  e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
>   src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
> PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> b469f9bbbdfbf98df947832411bd0cdce97affdc 
>   src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
> fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review125769
---



This patch does not apply cleanly against master (f28f41a), do you need to 
rebase?

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 28, 2016, 10:51 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 28, 2016, 10:51 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> ```json
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
> ```
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 6e9364e34db6dbb270468db3ff333b956c6bf9f3 
>   docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
>   examples/vagrant/announcer-auth.json PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py
>  e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
>   src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
> PRE-CREATION 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> b469f9bbbdfbf98df947832411bd0cdce97affdc 
>   src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
> fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Kunal Thakar



> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/bin/thermos_executor_main.py, line 
> > 107
> > 
> >
> > Sorry for not catching this earlier - how about s/auth/acl/?  I think 
> > this is more specific to what the file defines.

Kept it the same since we have more than ACLs in the file now.


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/bin/thermos_executor_main.py, line 
> > 111
> > 
> >
> > Is this more accurate? `Path to ZooKeeper ACL to use for announcer 
> > nodes.`

Done


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, line 84
> > 
> >
> > I think we should also require that at least one field is specified in 
> > `permissions`.  Seems like the ACL entry would be meaningliess otherwise.

Done. Will it be an overkill to use something like jsonschema to validate this 
instead?


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, lines 186-187
> > 
> >
> > Feeling some deja vu here - the double underscore is unconventional - 
> > please change to single underscore.

There are plenty of double underscores throughout the file that predate my 
changes. I am following the single underscore for new code. Should I refactor 
the entire file?


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, lines 95-96
> > 
> >
> > I suggest you just have `validate` raise for invalid and catch here, 
> > rather than returning a `bool`.  The sequence here creates two separate log 
> > entries that could just as easily be 1, e.g.
> > 
> > ```
> > Expecting a list of ACL objects
> > ZK authentication config is invalid
> > ```

Done


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, line 192
> > 
> >
> > I just realized something - does it make much sense to specify 
> > `default_acl` without also specifying `auth_data`?
> > 
> > ```
> > :param auth_data:
> >   A list of authentication credentials to use for the
> >   connection. Should be a list of (scheme, credential)
> >   tuples as :meth:`add_auth` takes.
> > ```
> > 
> > Worse yet - can the announcer lock itself out if it _doesn't_ 
> > authenticate itself?
> > 
> > Enabling this feature in the default vagrant environment will be 
> > informative, and may guide our approach.  But i do think that either way it 
> > makes sense to follow up with plumbing for `auth_data`.

Great catch.


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/test/python/apache/aurora/executor/common/test_announcer.py, line 352
> > 
> >
> > I assume this string is effectively a blob for this test.  If so, can 
> > you make it explicit by using a fake value (e.g. `'fake'`)?

Done


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > src/main/python/apache/aurora/executor/common/announcer.py, line 193
> > 
> >
> > Formatting nit - when a method signature or method call needs to wrap, 
> > please wrap to put each argument on its own line.  In this case:
> > 
> > ```
> > return KazooClient(
> > self._ensemble,
> > connection_retry=self.DEFAULT_RETRY_POLICY,
> > default_acl=self._zk_acl)
> > ```

Done


> On March 23, 2016, 2:23 a.m., Bill Farner wrote:
> > docs/security.md, line 289
> > 
> >
> > I'd like to propose several changes to this section, which i've made in 
> > the rewritten block below.
> > 
> > - Use consistent naming and proper nouns for projects (Thermos, 
> > ZooKeeper)
> > - Link to latest version of ZooKeeper docs
> > - Immediately link to relevant ZooKeeper ACL section
> > - Describe how to enable the feature before describing the format of 
> > the ACL file
> > - Use more accurate requirements level terminology, e.g. 
> > must/may/should (context reading http://tools.ietf.org/html/rfc2119)
> > 
> > ```
> > # Announcer Authentication
> > Nodes created by the Thermos executor may include ZooKeeper
> > 
> >

Re: Review Request 45042: Add ACL support for announcer

2016-03-28 Thread Kunal Thakar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/
---

(Updated March 28, 2016, 10:51 p.m.)


Review request for Aurora, Bill Farner and Zameer Manji.


Changes
---

Thanks for the feedback, auth_data is indeed a required piece of information 
that is needed before setting ACLs. I have updated the config file structure as 
follows to accomodate authentication credentials in addition to the ACL objects:
```json
{
  "auth": [
{
  "scheme": "",
  "credential": ""
}
  ],
  "acls": [
{
  "scheme": "",
  "credential": "",
  "permissions": {
"read": ,
"write": ,
"create": ,
"delete": ,
"admin": ,
"all": 
  }
}
  ]
}
```


Repository: aurora


Description
---

Add ACL support for announcer
https://issues.apache.org/jira/browse/AURORA-1643

Adding support for service discovery ZK authentication. ZK authentication 
secrets should be stored in a file as json (as follows):
```json
{
  "scheme": "",
  "credential": "",
"permissions": {
  "read": ,
  "write": ,
  "create": ,
  "delete": ,
  "admin": ,
  "all": 
}
}
```


Diffs (updated)
-

  RELEASE-NOTES.md 6e9364e34db6dbb270468db3ff333b956c6bf9f3 
  docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
  examples/vagrant/announcer-auth.json PRE-CREATION 
  examples/vagrant/upstart/aurora-scheduler-announcer-auth.conf PRE-CREATION 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
6634506108c346f8c23b2da7cc8d20d09d07d590 
  src/main/python/apache/aurora/executor/common/announcer.py 
79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
  
src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 
e9f7851292aef3a36da5da9b0fc333a7e7750cf3 
  src/test/python/apache/aurora/executor/common/test_announcer.py 
142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
  src/test/sh/org/apache/aurora/e2e/test_announcer_auth_end_to_end.sh 
PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
b469f9bbbdfbf98df947832411bd0cdce97affdc 
  src/test/sh/org/apache/aurora/e2e/validate_serverset.py 
fca1137bd2e7b1306a03dc2a54d2ef15b59af6a8 

Diff: https://reviews.apache.org/r/45042/diff/


Testing
---


Thanks,

Kunal Thakar

Re: Review Request 45222: Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in TaskConfig.

2016-03-28 Thread Amol Deshmukh



> On March 28, 2016, 1:23 p.m., Bill Farner wrote:
> > I won't have time to review this, so i'd like to stand down.
> > 
> > However, please make sure to add appropriate release notes, specifically 
> > for the stated backwards incompatibility.

Will do. I will post an updated review request with the release notes entry.


- Amol


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45222/#review125737
---


On March 28, 2016, 12:43 p.m., Amol Deshmukh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45222/
> ---
> 
> (Updated March 28, 2016, 12:43 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1616
> https://issues.apache.org/jira/browse/AURORA-1616
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Introduce "preemptible" flag in TierInfo with backward compatible support for 
> "production" flag in TaskConfig.
> 
> # Summary of changes
> - TierInfo extended to manage a new property: "preemptible"
> - If TaskConfig does not specify a tier, ``TierManager.getTier(..)`` falls 
> back to selecting a non-revocable tier matching ``TaskConfig.isProduction()``.
> - Removed ``TierInfo.DEFAULT`` and replaced its in test/benchmark code by 
> references to ``TaskTestUtil.DEV_TIER``.
> - Eager validation of tier specified in TaskConfig in 
> ``ConfigurationManager.validateAndPopulate(..)``
> 
> # Note on backward incompatible change
> TaskConfigs with a tier name not matching any tiers defined in ``tiers.json`` 
> were silently assigned a default tier. With this change, attempting to 
> schedule tasks that specify non-existent tier names will result in an error 
> (see ``ConfigurationManager.validateAndPopulate(ITaskConfig config)``).
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/Offers.java 
> 055a2ffcb13c643a3086343e3fbf71545c5fb0a6 
>   src/main/java/org/apache/aurora/scheduler/TierInfo.java 
> b4611b9cf99ca236cd1ea4610d01c0d293bf2e87 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java 
> 480c4853a6c87dd63a9810ae013e5cfacb29336d 
>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java 
> 97d87ff1b789625f2c07baf7a74a076f07742596 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 9cf8002e932d562e93fb8d17b4c56f564eb54ed5 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  b3b8ccf868c2a2f18f720a837e90d763072dd3eb 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> 16fa2d35cdf7ecdf82dd1ec7739e685ec1ff1aa2 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  7f84e90774193b0d31adb7dafcab0a249167cdba 
>   src/main/resources/org/apache/aurora/scheduler/tiers.json 
> 18701058076bedc5d1b667e2b97ad09ce84a9bb9 
>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java 
> 39096af816864ada32a9c07958975740e805f6b0 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 52113b80d91ecaf0cc2aeaad77e5fbc0ea4d1216 
>   src/test/java/org/apache/aurora/scheduler/ResourcesTest.java 
> 4fe8c518c580418078275b8056a5c195a765681e 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 
> a662100ba726cff0e47b4f9650753db9cecdef51 
>   src/test/java/org/apache/aurora/scheduler/TierModuleTest.java 
> e0601c83486671596e412f022ffae78b01c81c9d 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
>  1a520b3cb035a9afc25406d2f313c9f861eee4d6 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> 5103bd0f43d53079976b0f1596e299f2d91aa860 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  b6f5e4632ac1e028fdf93da1735463373e2d2788 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> b00add0b2fd4277e196505fffba4440e2e94207e 
>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 
> b1c71a6f1847d205c378d0b5a7269ea9d1165be5 
> 
> Diff: https://reviews.apache.org/r/45222/diff/
> 
> 
> Testing
> ---
> 
> # End to end tests
> ```
> $ ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ...
> *** OK (All tests passed) ***
> ```
> 
> # Jenkins build.sh
> ```
> $ ./build-support/jenkins/build.sh
> ...
>SUCCESS
> ```
> 
> # Local Scheduler
> ```
> $ ./gradlew run
> ...
> I0325 16:00:55.374 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> I0325 16:01:00.371 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> ```
> 
> # Attempting to schedule job with invalid tier-name
> ```
> vagrant@aurora:~/workspace$ aurora

Re: Review Request 45392: Reorganize Documentation

2016-03-28 Thread Stephan Erb



> On March 28, 2016, 10:10 p.m., Bill Farner wrote:
> > docs/installing.md, line 115
> > 
> >
> > I'm not a fan of the syntax, but the trailing WS here is necessary to 
> > force a line break.  Feel free to reformat (e.g. add another line break in 
> > the .md) to remove the need for the trailing WS though.
> > 
> > You can see the difference here.
> > 
> > https://github.com/apache/aurora/blob/master/docs/installing.md#centos-7-1
> > 
> > 
> > https://github.com/StephanErb/aurora/blob/userguide_docu/docs/operations/installation.md#centos-7

Good catch. I've added this to the list of minor glitches that need fixing. 
Will have that pull request ready in a day or two.


- Stephan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45392/#review125731
---


On March 28, 2016, 10:50 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45392/
> ---
> 
> (Updated March 28, 2016, 10:50 p.m.)
> 
> 
> Review request for Aurora, Steve Niemitz and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This started as a spike to structure the documentation in a way that makes it 
> more approachable. In addition, I believe the new structure will allow us to 
> extend and improve the documentation more easily, as the different sections 
> have more room to grow into something useful (eg. service discovery).
> 
> The new structure was inspired by the documentation of Hubspot's Singularity 
> scheduler. What I have done was mostly to cut & paste documentation and code 
> examples and embedded those into the following:
> 
> * getting-started: the most basic information for all users
> * features: proper explanation of our most important features. This should 
> make it much easier for people to discover Aurora's unique selling points.
> * operators: stuff only the operatos will care about
> * developers: stuff only contributors and committers acare about.
> * references: the details.
> 
> The diff is almost impossible review as I was not very diligent in keeping it 
> clean. Sorry for that. If you believe this is unacceptable, we can move to a 
> more piecewise review.
> 
> 
> Diffs
> -
> 
>   README.md e2b5632d42129d448d4fc237da9d939ca9dd564c 
>   docs/README.md 673c854825a0443a3bc7b13c04640dd85ad147f5 
>   docs/build-system.md 39c231dc9e222ebf409280199fe12d75449157fe 
>   docs/client-cluster-configuration.md 
> 88b9d772c9140178cd195875136408399e0ee52b 
>   docs/client-commands.md 156fe4cf2462735218bcdc80655ce3ec5e9cd4f5 
>   docs/committers.md f69a898d434abb0295e68682e290bc3f966d3654 
>   docs/configuration-reference.md 7bcf22d6ec8266568a1e3692fdbc451a58388967 
>   docs/configuration-tutorial.md 97664f39a5b2fce8c4829d5ba1c3ec7f4b8fc63b 
>   docs/cron-jobs.md 0f9842529ad075c9fbd153a13c2b07c94c391052 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   docs/design-documents.md 4d14caa4dde1a48d8d90b6b7c9104efd629d715c 
>   docs/design/command-hooks.md  
>   docs/developing-aurora-client.md 27f1c97c1b7c199c1c496e11ef8995ca3b1b437c 
>   docs/developing-aurora-scheduler.md 
> a703871a68b51e43cb51379df0856911dbe91c8b 
>   docs/development/thermos.md PRE-CREATION 
>   docs/development/ui.md PRE-CREATION 
>   docs/features/constraints.md PRE-CREATION 
>   docs/features/containers.md PRE-CREATION 
>   docs/features/job-updates.md PRE-CREATION 
>   docs/features/multitenancy.md PRE-CREATION 
>   docs/features/service-discovery.md PRE-CREATION 
>   docs/features/services.md PRE-CREATION 
>   docs/getting-started/overview.md PRE-CREATION 
>   docs/hooks.md 28307ab3d45721ccc97a3d01148129c22d1f1998 
>   docs/installing.md 5cc153ab8c7c8043efd07a7cb9fa44c84e31a7a8 
>   docs/monitoring.md  
>   docs/operations/configuration.md PRE-CREATION 
>   docs/presentations.md 84756a24e258a38cebad1dac7c6e51f7d6d187bd 
>   docs/reference/configuration-best-practices.md PRE-CREATION 
>   docs/reference/configuration-templating.md PRE-CREATION 
>   docs/reference/configuration-tutorial.md PRE-CREATION 
>   docs/reference/scheduler-configuration.md PRE-CREATION 
>   docs/resources.md 27a267828d8222623ae77e084e1d5f602075838e 
>   docs/scheduler-configuration.md 7e3d801251262a5eedbca8890316aa9f44be8314 
>   docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
>   docs/sla.md a558e0098e73172801d9fad927f1f85d0476413f 
>   docs/storage-config.md 7c6484183ce88eca6ab1a25805c86af1fcd64ee1 
>   docs/storage.md 6ffed541d1579a8131ce642b082be602431329d3 
>   docs/task-lifecycle.md 5d6456cb6647acb2c2c0f9024c744e37ccc55d1c 
>   docs/test-resource-generation.md

Re: Review Request 45392: Reorganize Documentation

2016-03-28 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45392/#review125742
---


Ship it!




Master (0950095) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 28, 2016, 6:12 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45392/
> ---
> 
> (Updated March 28, 2016, 6:12 p.m.)
> 
> 
> Review request for Aurora, John Sirois and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This started as a spike to structure the documentation in a way that makes it 
> more approachable. In addition, I believe the new structure will allow us to 
> extend and improve the documentation more easily, as the different sections 
> have more room to grow into something useful (eg. service discovery).
> 
> The new structure was inspired by the documentation of Hubspot's Singularity 
> scheduler. What I have done was mostly to cut & paste documentation and code 
> examples and embedded those into the following:
> 
> * getting-started: the most basic information for all users
> * features: proper explanation of our most important features. This should 
> make it much easier for people to discover Aurora's unique selling points.
> * operators: stuff only the operatos will care about
> * developers: stuff only contributors and committers acare about.
> * references: the details.
> 
> The diff is almost impossible review as I was not very diligent in keeping it 
> clean. Sorry for that. If you believe this is unacceptable, we can move to a 
> more piecewise review.
> 
> 
> Diffs
> -
> 
>   README.md e2b5632d42129d448d4fc237da9d939ca9dd564c 
>   docs/README.md 673c854825a0443a3bc7b13c04640dd85ad147f5 
>   docs/build-system.md 39c231dc9e222ebf409280199fe12d75449157fe 
>   docs/client-cluster-configuration.md 
> 88b9d772c9140178cd195875136408399e0ee52b 
>   docs/client-commands.md 156fe4cf2462735218bcdc80655ce3ec5e9cd4f5 
>   docs/committers.md f69a898d434abb0295e68682e290bc3f966d3654 
>   docs/configuration-reference.md 7bcf22d6ec8266568a1e3692fdbc451a58388967 
>   docs/configuration-tutorial.md 97664f39a5b2fce8c4829d5ba1c3ec7f4b8fc63b 
>   docs/cron-jobs.md 0f9842529ad075c9fbd153a13c2b07c94c391052 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   docs/design-documents.md 4d14caa4dde1a48d8d90b6b7c9104efd629d715c 
>   docs/design/command-hooks.md  
>   docs/developing-aurora-client.md 27f1c97c1b7c199c1c496e11ef8995ca3b1b437c 
>   docs/developing-aurora-scheduler.md 
> a703871a68b51e43cb51379df0856911dbe91c8b 
>   docs/development/thermos.md PRE-CREATION 
>   docs/development/ui.md PRE-CREATION 
>   docs/features/constraints.md PRE-CREATION 
>   docs/features/containers.md PRE-CREATION 
>   docs/features/job-updates.md PRE-CREATION 
>   docs/features/multitenancy.md PRE-CREATION 
>   docs/features/service-discovery.md PRE-CREATION 
>   docs/features/services.md PRE-CREATION 
>   docs/getting-started/overview.md PRE-CREATION 
>   docs/hooks.md 28307ab3d45721ccc97a3d01148129c22d1f1998 
>   docs/installing.md 5cc153ab8c7c8043efd07a7cb9fa44c84e31a7a8 
>   docs/monitoring.md  
>   docs/operations/configuration.md PRE-CREATION 
>   docs/presentations.md 84756a24e258a38cebad1dac7c6e51f7d6d187bd 
>   docs/reference/configuration-best-practices.md PRE-CREATION 
>   docs/reference/configuration-templating.md PRE-CREATION 
>   docs/reference/configuration-tutorial.md PRE-CREATION 
>   docs/reference/scheduler-configuration.md PRE-CREATION 
>   docs/resources.md 27a267828d8222623ae77e084e1d5f602075838e 
>   docs/scheduler-configuration.md 7e3d801251262a5eedbca8890316aa9f44be8314 
>   docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
>   docs/sla.md a558e0098e73172801d9fad927f1f85d0476413f 
>   docs/storage-config.md 7c6484183ce88eca6ab1a25805c86af1fcd64ee1 
>   docs/storage.md 6ffed541d1579a8131ce642b082be602431329d3 
>   docs/task-lifecycle.md 5d6456cb6647acb2c2c0f9024c744e37ccc55d1c 
>   docs/test-resource-generation.md e78e742542508389e8161d00ee0e4ddb7084e72d 
>   docs/thrift-deprecation.md 62a71bcf10704c92fd26a8fd74729a8dc51b988c 
>   docs/tools.md 2ae550db35835ac2ebe5a95dbcc18e86d7bc4642 
>   docs/tutorial.md 95539ef299792068de2ad9d38b39d66e162fbc21 
>   docs/user-guide.md 656296cc84622d44d6dd43fc1496496da15b5d08 
>   docs/vagrant.md 3bc201f17977dbc402eeac307500db5cc9d6bb27 
> 
> Diff: https://reviews.apache.org/r/45392/diff/
> 
> 
> Testing
> ---
> 
> Rendered version available at: 
> https://github.com/StephanErb/aurora/tree/userguide_docu/docs
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 45222: Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in TaskConfig.

2016-03-28 Thread Bill Farner


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45222/#review125737
---



I won't have time to review this, so i'd like to stand down.

However, please make sure to add appropriate release notes, specifically for 
the stated backwards incompatibility.

- Bill Farner


On March 28, 2016, 12:43 p.m., Amol Deshmukh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45222/
> ---
> 
> (Updated March 28, 2016, 12:43 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1616
> https://issues.apache.org/jira/browse/AURORA-1616
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Introduce "preemptible" flag in TierInfo with backward compatible support for 
> "production" flag in TaskConfig.
> 
> # Summary of changes
> - TierInfo extended to manage a new property: "preemptible"
> - If TaskConfig does not specify a tier, ``TierManager.getTier(..)`` falls 
> back to selecting a non-revocable tier matching ``TaskConfig.isProduction()``.
> - Removed ``TierInfo.DEFAULT`` and replaced its in test/benchmark code by 
> references to ``TaskTestUtil.DEV_TIER``.
> - Eager validation of tier specified in TaskConfig in 
> ``ConfigurationManager.validateAndPopulate(..)``
> 
> # Note on backward incompatible change
> TaskConfigs with a tier name not matching any tiers defined in ``tiers.json`` 
> were silently assigned a default tier. With this change, attempting to 
> schedule tasks that specify non-existent tier names will result in an error 
> (see ``ConfigurationManager.validateAndPopulate(ITaskConfig config)``).
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/Offers.java 
> 055a2ffcb13c643a3086343e3fbf71545c5fb0a6 
>   src/main/java/org/apache/aurora/scheduler/TierInfo.java 
> b4611b9cf99ca236cd1ea4610d01c0d293bf2e87 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java 
> 480c4853a6c87dd63a9810ae013e5cfacb29336d 
>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java 
> 97d87ff1b789625f2c07baf7a74a076f07742596 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 9cf8002e932d562e93fb8d17b4c56f564eb54ed5 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  b3b8ccf868c2a2f18f720a837e90d763072dd3eb 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> 16fa2d35cdf7ecdf82dd1ec7739e685ec1ff1aa2 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  7f84e90774193b0d31adb7dafcab0a249167cdba 
>   src/main/resources/org/apache/aurora/scheduler/tiers.json 
> 18701058076bedc5d1b667e2b97ad09ce84a9bb9 
>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java 
> 39096af816864ada32a9c07958975740e805f6b0 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 52113b80d91ecaf0cc2aeaad77e5fbc0ea4d1216 
>   src/test/java/org/apache/aurora/scheduler/ResourcesTest.java 
> 4fe8c518c580418078275b8056a5c195a765681e 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 
> a662100ba726cff0e47b4f9650753db9cecdef51 
>   src/test/java/org/apache/aurora/scheduler/TierModuleTest.java 
> e0601c83486671596e412f022ffae78b01c81c9d 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
>  1a520b3cb035a9afc25406d2f313c9f861eee4d6 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> 5103bd0f43d53079976b0f1596e299f2d91aa860 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  b6f5e4632ac1e028fdf93da1735463373e2d2788 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> b00add0b2fd4277e196505fffba4440e2e94207e 
>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 
> b1c71a6f1847d205c378d0b5a7269ea9d1165be5 
> 
> Diff: https://reviews.apache.org/r/45222/diff/
> 
> 
> Testing
> ---
> 
> # End to end tests
> ```
> $ ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ...
> *** OK (All tests passed) ***
> ```
> 
> # Jenkins build.sh
> ```
> $ ./build-support/jenkins/build.sh
> ...
>SUCCESS
> ```
> 
> # Local Scheduler
> ```
> $ ./gradlew run
> ...
> I0325 16:00:55.374 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> I0325 16:01:00.371 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> ```
> 
> # Attempting to schedule job with invalid tier-name
> ```
> vagrant@aurora:~/workspace$ aurora job create devcluster/vagrant/devel/test 
> job.aurora
>  INFO] Creating job test
> Job creation failed due to error:
>   Invalid tier

Re: Review Request 45392: Reorganize Documentation

2016-03-28 Thread Steve Niemitz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45392/#review125734
---


Ship it!




Ship It!

- Steve Niemitz


On March 28, 2016, 6:12 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45392/
> ---
> 
> (Updated March 28, 2016, 6:12 p.m.)
> 
> 
> Review request for Aurora, John Sirois and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This started as a spike to structure the documentation in a way that makes it 
> more approachable. In addition, I believe the new structure will allow us to 
> extend and improve the documentation more easily, as the different sections 
> have more room to grow into something useful (eg. service discovery).
> 
> The new structure was inspired by the documentation of Hubspot's Singularity 
> scheduler. What I have done was mostly to cut & paste documentation and code 
> examples and embedded those into the following:
> 
> * getting-started: the most basic information for all users
> * features: proper explanation of our most important features. This should 
> make it much easier for people to discover Aurora's unique selling points.
> * operators: stuff only the operatos will care about
> * developers: stuff only contributors and committers acare about.
> * references: the details.
> 
> The diff is almost impossible review as I was not very diligent in keeping it 
> clean. Sorry for that. If you believe this is unacceptable, we can move to a 
> more piecewise review.
> 
> 
> Diffs
> -
> 
>   README.md e2b5632d42129d448d4fc237da9d939ca9dd564c 
>   docs/README.md 673c854825a0443a3bc7b13c04640dd85ad147f5 
>   docs/build-system.md 39c231dc9e222ebf409280199fe12d75449157fe 
>   docs/client-cluster-configuration.md 
> 88b9d772c9140178cd195875136408399e0ee52b 
>   docs/client-commands.md 156fe4cf2462735218bcdc80655ce3ec5e9cd4f5 
>   docs/committers.md f69a898d434abb0295e68682e290bc3f966d3654 
>   docs/configuration-reference.md 7bcf22d6ec8266568a1e3692fdbc451a58388967 
>   docs/configuration-tutorial.md 97664f39a5b2fce8c4829d5ba1c3ec7f4b8fc63b 
>   docs/cron-jobs.md 0f9842529ad075c9fbd153a13c2b07c94c391052 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   docs/design-documents.md 4d14caa4dde1a48d8d90b6b7c9104efd629d715c 
>   docs/design/command-hooks.md  
>   docs/developing-aurora-client.md 27f1c97c1b7c199c1c496e11ef8995ca3b1b437c 
>   docs/developing-aurora-scheduler.md 
> a703871a68b51e43cb51379df0856911dbe91c8b 
>   docs/development/thermos.md PRE-CREATION 
>   docs/development/ui.md PRE-CREATION 
>   docs/features/constraints.md PRE-CREATION 
>   docs/features/containers.md PRE-CREATION 
>   docs/features/job-updates.md PRE-CREATION 
>   docs/features/multitenancy.md PRE-CREATION 
>   docs/features/service-discovery.md PRE-CREATION 
>   docs/features/services.md PRE-CREATION 
>   docs/getting-started/overview.md PRE-CREATION 
>   docs/hooks.md 28307ab3d45721ccc97a3d01148129c22d1f1998 
>   docs/installing.md 5cc153ab8c7c8043efd07a7cb9fa44c84e31a7a8 
>   docs/monitoring.md  
>   docs/operations/configuration.md PRE-CREATION 
>   docs/presentations.md 84756a24e258a38cebad1dac7c6e51f7d6d187bd 
>   docs/reference/configuration-best-practices.md PRE-CREATION 
>   docs/reference/configuration-templating.md PRE-CREATION 
>   docs/reference/configuration-tutorial.md PRE-CREATION 
>   docs/reference/scheduler-configuration.md PRE-CREATION 
>   docs/resources.md 27a267828d8222623ae77e084e1d5f602075838e 
>   docs/scheduler-configuration.md 7e3d801251262a5eedbca8890316aa9f44be8314 
>   docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
>   docs/sla.md a558e0098e73172801d9fad927f1f85d0476413f 
>   docs/storage-config.md 7c6484183ce88eca6ab1a25805c86af1fcd64ee1 
>   docs/storage.md 6ffed541d1579a8131ce642b082be602431329d3 
>   docs/task-lifecycle.md 5d6456cb6647acb2c2c0f9024c744e37ccc55d1c 
>   docs/test-resource-generation.md e78e742542508389e8161d00ee0e4ddb7084e72d 
>   docs/thrift-deprecation.md 62a71bcf10704c92fd26a8fd74729a8dc51b988c 
>   docs/tools.md 2ae550db35835ac2ebe5a95dbcc18e86d7bc4642 
>   docs/tutorial.md 95539ef299792068de2ad9d38b39d66e162fbc21 
>   docs/user-guide.md 656296cc84622d44d6dd43fc1496496da15b5d08 
>   docs/vagrant.md 3bc201f17977dbc402eeac307500db5cc9d6bb27 
> 
> Diff: https://reviews.apache.org/r/45392/diff/
> 
> 
> Testing
> ---
> 
> Rendered version available at: 
> https://github.com/StephanErb/aurora/tree/userguide_docu/docs
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 45392: Reorganize Documentation

2016-03-28 Thread Stephan Erb


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45392/#review125730
---



@ReviewBot retry

- Stephan Erb


On March 28, 2016, 8:12 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45392/
> ---
> 
> (Updated March 28, 2016, 8:12 p.m.)
> 
> 
> Review request for Aurora, John Sirois and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> This started as a spike to structure the documentation in a way that makes it 
> more approachable. In addition, I believe the new structure will allow us to 
> extend and improve the documentation more easily, as the different sections 
> have more room to grow into something useful (eg. service discovery).
> 
> The new structure was inspired by the documentation of Hubspot's Singularity 
> scheduler. What I have done was mostly to cut & paste documentation and code 
> examples and embedded those into the following:
> 
> * getting-started: the most basic information for all users
> * features: proper explanation of our most important features. This should 
> make it much easier for people to discover Aurora's unique selling points.
> * operators: stuff only the operatos will care about
> * developers: stuff only contributors and committers acare about.
> * references: the details.
> 
> The diff is almost impossible review as I was not very diligent in keeping it 
> clean. Sorry for that. If you believe this is unacceptable, we can move to a 
> more piecewise review.
> 
> 
> Diffs
> -
> 
>   README.md e2b5632d42129d448d4fc237da9d939ca9dd564c 
>   docs/README.md 673c854825a0443a3bc7b13c04640dd85ad147f5 
>   docs/build-system.md 39c231dc9e222ebf409280199fe12d75449157fe 
>   docs/client-cluster-configuration.md 
> 88b9d772c9140178cd195875136408399e0ee52b 
>   docs/client-commands.md 156fe4cf2462735218bcdc80655ce3ec5e9cd4f5 
>   docs/committers.md f69a898d434abb0295e68682e290bc3f966d3654 
>   docs/configuration-reference.md 7bcf22d6ec8266568a1e3692fdbc451a58388967 
>   docs/configuration-tutorial.md 97664f39a5b2fce8c4829d5ba1c3ec7f4b8fc63b 
>   docs/cron-jobs.md 0f9842529ad075c9fbd153a13c2b07c94c391052 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   docs/design-documents.md 4d14caa4dde1a48d8d90b6b7c9104efd629d715c 
>   docs/design/command-hooks.md  
>   docs/developing-aurora-client.md 27f1c97c1b7c199c1c496e11ef8995ca3b1b437c 
>   docs/developing-aurora-scheduler.md 
> a703871a68b51e43cb51379df0856911dbe91c8b 
>   docs/development/thermos.md PRE-CREATION 
>   docs/development/ui.md PRE-CREATION 
>   docs/features/constraints.md PRE-CREATION 
>   docs/features/containers.md PRE-CREATION 
>   docs/features/job-updates.md PRE-CREATION 
>   docs/features/multitenancy.md PRE-CREATION 
>   docs/features/service-discovery.md PRE-CREATION 
>   docs/features/services.md PRE-CREATION 
>   docs/getting-started/overview.md PRE-CREATION 
>   docs/hooks.md 28307ab3d45721ccc97a3d01148129c22d1f1998 
>   docs/installing.md 5cc153ab8c7c8043efd07a7cb9fa44c84e31a7a8 
>   docs/monitoring.md  
>   docs/operations/configuration.md PRE-CREATION 
>   docs/presentations.md 84756a24e258a38cebad1dac7c6e51f7d6d187bd 
>   docs/reference/configuration-best-practices.md PRE-CREATION 
>   docs/reference/configuration-templating.md PRE-CREATION 
>   docs/reference/configuration-tutorial.md PRE-CREATION 
>   docs/reference/scheduler-configuration.md PRE-CREATION 
>   docs/resources.md 27a267828d8222623ae77e084e1d5f602075838e 
>   docs/scheduler-configuration.md 7e3d801251262a5eedbca8890316aa9f44be8314 
>   docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
>   docs/sla.md a558e0098e73172801d9fad927f1f85d0476413f 
>   docs/storage-config.md 7c6484183ce88eca6ab1a25805c86af1fcd64ee1 
>   docs/storage.md 6ffed541d1579a8131ce642b082be602431329d3 
>   docs/task-lifecycle.md 5d6456cb6647acb2c2c0f9024c744e37ccc55d1c 
>   docs/test-resource-generation.md e78e742542508389e8161d00ee0e4ddb7084e72d 
>   docs/thrift-deprecation.md 62a71bcf10704c92fd26a8fd74729a8dc51b988c 
>   docs/tools.md 2ae550db35835ac2ebe5a95dbcc18e86d7bc4642 
>   docs/tutorial.md 95539ef299792068de2ad9d38b39d66e162fbc21 
>   docs/user-guide.md 656296cc84622d44d6dd43fc1496496da15b5d08 
>   docs/vagrant.md 3bc201f17977dbc402eeac307500db5cc9d6bb27 
> 
> Diff: https://reviews.apache.org/r/45392/diff/
> 
> 
> Testing
> ---
> 
> Rendered version available at: 
> https://github.com/StephanErb/aurora/tree/userguide_docu/docs
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 45222: Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in TaskConfig.

2016-03-28 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45222/#review125729
---



This patch does not apply cleanly against master (0950095), do you need to 
rebase?

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 28, 2016, 7:43 p.m., Amol Deshmukh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45222/
> ---
> 
> (Updated March 28, 2016, 7:43 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1616
> https://issues.apache.org/jira/browse/AURORA-1616
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Introduce "preemptible" flag in TierInfo with backward compatible support for 
> "production" flag in TaskConfig.
> 
> # Summary of changes
> - TierInfo extended to manage a new property: "preemptible"
> - If TaskConfig does not specify a tier, ``TierManager.getTier(..)`` falls 
> back to selecting a non-revocable tier matching ``TaskConfig.isProduction()``.
> - Removed ``TierInfo.DEFAULT`` and replaced its in test/benchmark code by 
> references to ``TaskTestUtil.DEV_TIER``.
> - Eager validation of tier specified in TaskConfig in 
> ``ConfigurationManager.validateAndPopulate(..)``
> 
> # Note on backward incompatible change
> TaskConfigs with a tier name not matching any tiers defined in ``tiers.json`` 
> were silently assigned a default tier. With this change, attempting to 
> schedule tasks that specify non-existent tier names will result in an error 
> (see ``ConfigurationManager.validateAndPopulate(ITaskConfig config)``).
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/Offers.java 
> 055a2ffcb13c643a3086343e3fbf71545c5fb0a6 
>   src/main/java/org/apache/aurora/scheduler/TierInfo.java 
> b4611b9cf99ca236cd1ea4610d01c0d293bf2e87 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java 
> 480c4853a6c87dd63a9810ae013e5cfacb29336d 
>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java 
> 97d87ff1b789625f2c07baf7a74a076f07742596 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 9cf8002e932d562e93fb8d17b4c56f564eb54ed5 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  b3b8ccf868c2a2f18f720a837e90d763072dd3eb 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> 16fa2d35cdf7ecdf82dd1ec7739e685ec1ff1aa2 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  7f84e90774193b0d31adb7dafcab0a249167cdba 
>   src/main/resources/org/apache/aurora/scheduler/tiers.json 
> 18701058076bedc5d1b667e2b97ad09ce84a9bb9 
>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java 
> 39096af816864ada32a9c07958975740e805f6b0 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 52113b80d91ecaf0cc2aeaad77e5fbc0ea4d1216 
>   src/test/java/org/apache/aurora/scheduler/ResourcesTest.java 
> 4fe8c518c580418078275b8056a5c195a765681e 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 
> a662100ba726cff0e47b4f9650753db9cecdef51 
>   src/test/java/org/apache/aurora/scheduler/TierModuleTest.java 
> e0601c83486671596e412f022ffae78b01c81c9d 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
>  1a520b3cb035a9afc25406d2f313c9f861eee4d6 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> 5103bd0f43d53079976b0f1596e299f2d91aa860 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  b6f5e4632ac1e028fdf93da1735463373e2d2788 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> b00add0b2fd4277e196505fffba4440e2e94207e 
>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 
> b1c71a6f1847d205c378d0b5a7269ea9d1165be5 
> 
> Diff: https://reviews.apache.org/r/45222/diff/
> 
> 
> Testing
> ---
> 
> # End to end tests
> ```
> $ ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ...
> *** OK (All tests passed) ***
> ```
> 
> # Jenkins build.sh
> ```
> $ ./build-support/jenkins/build.sh
> ...
>SUCCESS
> ```
> 
> # Local Scheduler
> ```
> $ ./gradlew run
> ...
> I0325 16:00:55.374 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> I0325 16:01:00.371 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> ```
> 
> # Attempting to schedule job with invalid tier-name
> ```
> vagrant@aurora:~/workspace$ aurora job create devcluster/vagrant/devel/test 
> job.aurora
>  INFO] Creating job test
> Job creation failed due to error:
>   Invalid tier

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/#review125728
---



Master (0950095) is green with this patch.
  ./build-support/jenkins/build.sh

However, it appears that it might lack test coverage.

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 28, 2016, 6:21 p.m., George Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45396/
> ---
> 
> (Updated March 28, 2016, 6:21 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> In instances where the root filesystem is read-only, it is desirable to
> have the executor/runner extract themselves into the sandbox.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  949c299bdbc54f976db994266fb97f3099256f13 
> 
> Diff: https://reviews.apache.org/r/45396/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> George Sirois
> 
>

Re: Review Request 45222: Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in TaskConfig.

2016-03-28 Thread Amol Deshmukh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45222/
---

(Updated March 28, 2016, 12:43 p.m.)


Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner.


Changes
---

- Optimized the iteration of victim tasks.
- Updated expected frequency of the ``tierManager.getTier(..)`` mock in unit 
tests to exact value (with couple of exceptions for reasons mentioned in the 
previous comment).


Bugs: AURORA-1616
https://issues.apache.org/jira/browse/AURORA-1616


Repository: aurora


Description
---

Introduce "preemptible" flag in TierInfo with backward compatible support for 
"production" flag in TaskConfig.

# Summary of changes
- TierInfo extended to manage a new property: "preemptible"
- If TaskConfig does not specify a tier, ``TierManager.getTier(..)`` falls back 
to selecting a non-revocable tier matching ``TaskConfig.isProduction()``.
- Removed ``TierInfo.DEFAULT`` and replaced its in test/benchmark code by 
references to ``TaskTestUtil.DEV_TIER``.
- Eager validation of tier specified in TaskConfig in 
``ConfigurationManager.validateAndPopulate(..)``

# Note on backward incompatible change
TaskConfigs with a tier name not matching any tiers defined in ``tiers.json`` 
were silently assigned a default tier. With this change, attempting to schedule 
tasks that specify non-existent tier names will result in an error (see 
``ConfigurationManager.validateAndPopulate(ITaskConfig config)``).


Diffs (updated)
-

  src/jmh/java/org/apache/aurora/benchmark/Offers.java 
055a2ffcb13c643a3086343e3fbf71545c5fb0a6 
  src/main/java/org/apache/aurora/scheduler/TierInfo.java 
b4611b9cf99ca236cd1ea4610d01c0d293bf2e87 
  src/main/java/org/apache/aurora/scheduler/TierManager.java 
480c4853a6c87dd63a9810ae013e5cfacb29336d 
  src/main/java/org/apache/aurora/scheduler/app/AppModule.java 
97d87ff1b789625f2c07baf7a74a076f07742596 
  src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
9cf8002e932d562e93fb8d17b4c56f564eb54ed5 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 b3b8ccf868c2a2f18f720a837e90d763072dd3eb 
  src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
16fa2d35cdf7ecdf82dd1ec7739e685ec1ff1aa2 
  
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 
7f84e90774193b0d31adb7dafcab0a249167cdba 
  src/main/resources/org/apache/aurora/scheduler/tiers.json 
18701058076bedc5d1b667e2b97ad09ce84a9bb9 
  src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java 
39096af816864ada32a9c07958975740e805f6b0 
  src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
52113b80d91ecaf0cc2aeaad77e5fbc0ea4d1216 
  src/test/java/org/apache/aurora/scheduler/ResourcesTest.java 
4fe8c518c580418078275b8056a5c195a765681e 
  src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 
a662100ba726cff0e47b4f9650753db9cecdef51 
  src/test/java/org/apache/aurora/scheduler/TierModuleTest.java 
e0601c83486671596e412f022ffae78b01c81c9d 
  
src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
 1a520b3cb035a9afc25406d2f313c9f861eee4d6 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
5103bd0f43d53079976b0f1596e299f2d91aa860 
  
src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
 b6f5e4632ac1e028fdf93da1735463373e2d2788 
  src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
b00add0b2fd4277e196505fffba4440e2e94207e 
  src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 
b1c71a6f1847d205c378d0b5a7269ea9d1165be5 

Diff: https://reviews.apache.org/r/45222/diff/


Testing
---

# End to end tests
```
$ ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
...
*** OK (All tests passed) ***
```

# Jenkins build.sh
```
$ ./build-support/jenkins/build.sh
...
   SUCCESS
```

# Local Scheduler
```
$ ./gradlew run
...
I0325 16:00:55.374 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
suppressing offer cycle.
I0325 16:01:00.371 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
suppressing offer cycle.
```

# Attempting to schedule job with invalid tier-name
```
vagrant@aurora:~/workspace$ aurora job create devcluster/vagrant/devel/test 
job.aurora
 INFO] Creating job test
Job creation failed due to error:
Invalid tier 'badtier' in TaskConfig.
```


Thanks,

Amol Deshmukh

Re: Review Request 45222: Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in TaskConfig.

2016-03-28 Thread Amol Deshmukh



> On March 28, 2016, 9:36 a.m., Maxim Khutornenko wrote:
> > src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java,
> >  line 131
> > 
> >
> > Any reason to allow more than "once" (default) here? Same in other 
> > tests.
> 
> Amol Deshmukh wrote:
> The actual cardinality of the call ``tierManager.getTier(lowPriority)`` 
> here is 3:
> 1. Once in the ``victimToResources`` filter (existing behavior)
> 2. Twice in 
> ``o.a.a.s.preemptor.PreemptionVictimFilter.PreemptionVictimFilterImpl#filterPreemptionVictims``,
>  due to the lazy evaluation of the ``FluentIterable preemptableTasks``:
>   a. Once via ``preemptableTasks.isEmpty()``
>   b. Once via ``resourceOrder.immutableSortedCopy(preemptableTasks)``
> 
> Now that I think about it, it is just as easily possible to have the 
> FluentIterable evaluated once by making the ``immutableSortedCopy`` before 
> checking for emptiness. I will repost an updated review.

Ok, so while that optimization does cut down the number of times the filter 
operation is performed, there is still some indeterminism introduced due to the 
use of ``victimToResources`` transform in ``resourceOrder`` for tests involving 
more than one eligible victim. Those cases, I believe, will be best served by 
the use of ``atLeastOnce()`` instead of the exact cardinality discovered 
through testing.


- Amol


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45222/#review125671
---


On March 25, 2016, 5:08 p.m., Amol Deshmukh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45222/
> ---
> 
> (Updated March 25, 2016, 5:08 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1616
> https://issues.apache.org/jira/browse/AURORA-1616
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Introduce "preemptible" flag in TierInfo with backward compatible support for 
> "production" flag in TaskConfig.
> 
> # Summary of changes
> - TierInfo extended to manage a new property: "preemptible"
> - If TaskConfig does not specify a tier, ``TierManager.getTier(..)`` falls 
> back to selecting a non-revocable tier matching ``TaskConfig.isProduction()``.
> - Removed ``TierInfo.DEFAULT`` and replaced its in test/benchmark code by 
> references to ``TaskTestUtil.DEV_TIER``.
> - Eager validation of tier specified in TaskConfig in 
> ``ConfigurationManager.validateAndPopulate(..)``
> 
> # Note on backward incompatible change
> TaskConfigs with a tier name not matching any tiers defined in ``tiers.json`` 
> were silently assigned a default tier. With this change, attempting to 
> schedule tasks that specify non-existent tier names will result in an error 
> (see ``ConfigurationManager.validateAndPopulate(ITaskConfig config)``).
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/Offers.java 
> 055a2ffcb13c643a3086343e3fbf71545c5fb0a6 
>   src/main/java/org/apache/aurora/scheduler/TierInfo.java 
> b4611b9cf99ca236cd1ea4610d01c0d293bf2e87 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java 
> 480c4853a6c87dd63a9810ae013e5cfacb29336d 
>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java 
> 97d87ff1b789625f2c07baf7a74a076f07742596 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 9cf8002e932d562e93fb8d17b4c56f564eb54ed5 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  b3b8ccf868c2a2f18f720a837e90d763072dd3eb 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> 16fa2d35cdf7ecdf82dd1ec7739e685ec1ff1aa2 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  7f84e90774193b0d31adb7dafcab0a249167cdba 
>   src/main/resources/org/apache/aurora/scheduler/tiers.json 
> 18701058076bedc5d1b667e2b97ad09ce84a9bb9 
>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java 
> 39096af816864ada32a9c07958975740e805f6b0 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 52113b80d91ecaf0cc2aeaad77e5fbc0ea4d1216 
>   src/test/java/org/apache/aurora/scheduler/ResourcesTest.java 
> 4fe8c518c580418078275b8056a5c195a765681e 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 
> a662100ba726cff0e47b4f9650753db9cecdef51 
>   src/test/java/org/apache/aurora/scheduler/TierModuleTest.java 
> e0601c83486671596e412f022ffae78b01c81c9d 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
>  1a520b3cb035a9afc25406d2f313c9f861eee4d6 
>   
>

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread Steve Niemitz

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/#review125722
---

RELEASE-NOTES.md (line 14)

In the future I think we should revisit this and make this the default 
behavior.  The only reason I think it shouldn't be enabled by default now is 
because the mesos.native lib inside the pex is massive, and duplicating it into 
every sandbox is 1) slow, and 2) a waste of space.

However, once my patch lands, the lib goes down to ~8 MB, which I think is 
worth duplicating, especially to get rid of any weird race conditions with 
multiple executors launching.

- Steve Niemitz

On March 28, 2016, 6:21 p.m., George Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45396/
> ---
> 
> (Updated March 28, 2016, 6:21 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> In instances where the root filesystem is read-only, it is desirable to
> have the executor/runner extract themselves into the sandbox.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  949c299bdbc54f976db994266fb97f3099256f13 
> 
> Diff: https://reviews.apache.org/r/45396/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> George Sirois
> 
>

Re: Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread Steve Niemitz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/#review125726
---


Ship it!




Ship It!

- Steve Niemitz


On March 28, 2016, 6:21 p.m., George Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45396/
> ---
> 
> (Updated March 28, 2016, 6:21 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> In instances where the root filesystem is read-only, it is desirable to
> have the executor/runner extract themselves into the sandbox.
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
>   docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  949c299bdbc54f976db994266fb97f3099256f13 
> 
> Diff: https://reviews.apache.org/r/45396/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> George Sirois
> 
>

Review Request 45396: Adds the ability to set HOME to the sandbox before running the executor.

2016-03-28 Thread George Sirois


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45396/
---

Review request for Aurora, Joshua Cohen, Steve Niemitz, and Bill Farner.


Repository: aurora


Description
---

In instances where the root filesystem is read-only, it is desirable to
have the executor/runner extract themselves into the sandbox.


Diffs
-

  RELEASE-NOTES.md 34f28a165aae4ae24fa95ef19b4972e088fd63a0 
  docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
  
src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
 949c299bdbc54f976db994266fb97f3099256f13 

Diff: https://reviews.apache.org/r/45396/diff/


Testing
---


Thanks,

George Sirois

Re: Review Request 45222: Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in TaskConfig.

2016-03-28 Thread Amol Deshmukh



> On March 28, 2016, 9:36 a.m., Maxim Khutornenko wrote:
> > src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java,
> >  line 131
> > 
> >
> > Any reason to allow more than "once" (default) here? Same in other 
> > tests.

The actual cardinality of the call ``tierManager.getTier(lowPriority)`` here is 
3:
1. Once in the ``victimToResources`` filter (existing behavior)
2. Twice in 
``o.a.a.s.preemptor.PreemptionVictimFilter.PreemptionVictimFilterImpl#filterPreemptionVictims``,
 due to the lazy evaluation of the ``FluentIterable preemptableTasks``:
  a. Once via ``preemptableTasks.isEmpty()``
  b. Once via ``resourceOrder.immutableSortedCopy(preemptableTasks)``

Now that I think about it, it is just as easily possible to have the 
FluentIterable evaluated once by making the ``immutableSortedCopy`` before 
checking for emptiness. I will repost an updated review.


- Amol


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45222/#review125671
---


On March 25, 2016, 5:08 p.m., Amol Deshmukh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45222/
> ---
> 
> (Updated March 25, 2016, 5:08 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1616
> https://issues.apache.org/jira/browse/AURORA-1616
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Introduce "preemptible" flag in TierInfo with backward compatible support for 
> "production" flag in TaskConfig.
> 
> # Summary of changes
> - TierInfo extended to manage a new property: "preemptible"
> - If TaskConfig does not specify a tier, ``TierManager.getTier(..)`` falls 
> back to selecting a non-revocable tier matching ``TaskConfig.isProduction()``.
> - Removed ``TierInfo.DEFAULT`` and replaced its in test/benchmark code by 
> references to ``TaskTestUtil.DEV_TIER``.
> - Eager validation of tier specified in TaskConfig in 
> ``ConfigurationManager.validateAndPopulate(..)``
> 
> # Note on backward incompatible change
> TaskConfigs with a tier name not matching any tiers defined in ``tiers.json`` 
> were silently assigned a default tier. With this change, attempting to 
> schedule tasks that specify non-existent tier names will result in an error 
> (see ``ConfigurationManager.validateAndPopulate(ITaskConfig config)``).
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/Offers.java 
> 055a2ffcb13c643a3086343e3fbf71545c5fb0a6 
>   src/main/java/org/apache/aurora/scheduler/TierInfo.java 
> b4611b9cf99ca236cd1ea4610d01c0d293bf2e87 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java 
> 480c4853a6c87dd63a9810ae013e5cfacb29336d 
>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java 
> 97d87ff1b789625f2c07baf7a74a076f07742596 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 9cf8002e932d562e93fb8d17b4c56f564eb54ed5 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  b3b8ccf868c2a2f18f720a837e90d763072dd3eb 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> 16fa2d35cdf7ecdf82dd1ec7739e685ec1ff1aa2 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  7f84e90774193b0d31adb7dafcab0a249167cdba 
>   src/main/resources/org/apache/aurora/scheduler/tiers.json 
> 18701058076bedc5d1b667e2b97ad09ce84a9bb9 
>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java 
> 39096af816864ada32a9c07958975740e805f6b0 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 52113b80d91ecaf0cc2aeaad77e5fbc0ea4d1216 
>   src/test/java/org/apache/aurora/scheduler/ResourcesTest.java 
> 4fe8c518c580418078275b8056a5c195a765681e 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 
> a662100ba726cff0e47b4f9650753db9cecdef51 
>   src/test/java/org/apache/aurora/scheduler/TierModuleTest.java 
> e0601c83486671596e412f022ffae78b01c81c9d 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
>  1a520b3cb035a9afc25406d2f313c9f861eee4d6 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> 5103bd0f43d53079976b0f1596e299f2d91aa860 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  b6f5e4632ac1e028fdf93da1735463373e2d2788 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> b00add0b2fd4277e196505fffba4440e2e94207e 
>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 
> b1c71a6f1847d205c378d0b5a7269ea9d1165be5 
> 
> Diff:

Review Request 45392: Reorganize Documentation

2016-03-28 Thread Stephan Erb


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45392/
---

Review request for Aurora, John Sirois and Bill Farner.


Repository: aurora


Description
---

This started as a spike to structure the documentation in a way that makes it 
more approachable. In addition, I believe the new structure will allow us to 
extend and improve the documentation more easily, as the different sections 
have more room to grow into something useful (eg. service discovery).

The new structure was inspired by the documentation of Hubspot's Singularity 
scheduler. What I have done was mostly to cut & paste documentation and code 
examples and embedded those into the following:

* getting-started: the most basic information for all users
* features: proper explanation of our most important features. This should make 
it much easier for people to discover Aurora's unique selling points.
* operators: stuff only the operatos will care about
* developers: stuff only contributors and committers acare about.
* references: the details.

The diff is almost impossible review as I was not very diligent in keeping it 
clean. Sorry for that. If you believe this is unacceptable, we can move to a 
more piecewise review.


Diffs
-

  README.md e2b5632d42129d448d4fc237da9d939ca9dd564c 
  docs/README.md 673c854825a0443a3bc7b13c04640dd85ad147f5 
  docs/build-system.md 39c231dc9e222ebf409280199fe12d75449157fe 
  docs/client-cluster-configuration.md 88b9d772c9140178cd195875136408399e0ee52b 
  docs/client-commands.md 156fe4cf2462735218bcdc80655ce3ec5e9cd4f5 
  docs/committers.md f69a898d434abb0295e68682e290bc3f966d3654 
  docs/configuration-reference.md 7bcf22d6ec8266568a1e3692fdbc451a58388967 
  docs/configuration-tutorial.md 97664f39a5b2fce8c4829d5ba1c3ec7f4b8fc63b 
  docs/cron-jobs.md 0f9842529ad075c9fbd153a13c2b07c94c391052 
  docs/deploying-aurora-scheduler.md 03bfdbab927c924486b04c42df2ad236c0f414a0 
  docs/design-documents.md 4d14caa4dde1a48d8d90b6b7c9104efd629d715c 
  docs/design/command-hooks.md  
  docs/developing-aurora-client.md 27f1c97c1b7c199c1c496e11ef8995ca3b1b437c 
  docs/developing-aurora-scheduler.md a703871a68b51e43cb51379df0856911dbe91c8b 
  docs/development/thermos.md PRE-CREATION 
  docs/development/ui.md PRE-CREATION 
  docs/features/constraints.md PRE-CREATION 
  docs/features/containers.md PRE-CREATION 
  docs/features/job-updates.md PRE-CREATION 
  docs/features/multitenancy.md PRE-CREATION 
  docs/features/service-discovery.md PRE-CREATION 
  docs/features/services.md PRE-CREATION 
  docs/getting-started/overview.md PRE-CREATION 
  docs/hooks.md 28307ab3d45721ccc97a3d01148129c22d1f1998 
  docs/installing.md 5cc153ab8c7c8043efd07a7cb9fa44c84e31a7a8 
  docs/monitoring.md  
  docs/operations/configuration.md PRE-CREATION 
  docs/presentations.md 84756a24e258a38cebad1dac7c6e51f7d6d187bd 
  docs/reference/configuration-best-practices.md PRE-CREATION 
  docs/reference/configuration-templating.md PRE-CREATION 
  docs/reference/configuration-tutorial.md PRE-CREATION 
  docs/reference/scheduler-configuration.md PRE-CREATION 
  docs/resources.md 27a267828d8222623ae77e084e1d5f602075838e 
  docs/scheduler-configuration.md 7e3d801251262a5eedbca8890316aa9f44be8314 
  docs/security.md 32bea426fbceec83187e851a5db11e82df55e962 
  docs/sla.md a558e0098e73172801d9fad927f1f85d0476413f 
  docs/storage-config.md 7c6484183ce88eca6ab1a25805c86af1fcd64ee1 
  docs/storage.md 6ffed541d1579a8131ce642b082be602431329d3 
  docs/task-lifecycle.md 5d6456cb6647acb2c2c0f9024c744e37ccc55d1c 
  docs/test-resource-generation.md e78e742542508389e8161d00ee0e4ddb7084e72d 
  docs/thrift-deprecation.md 62a71bcf10704c92fd26a8fd74729a8dc51b988c 
  docs/tools.md 2ae550db35835ac2ebe5a95dbcc18e86d7bc4642 
  docs/tutorial.md 95539ef299792068de2ad9d38b39d66e162fbc21 
  docs/user-guide.md 656296cc84622d44d6dd43fc1496496da15b5d08 
  docs/vagrant.md 3bc201f17977dbc402eeac307500db5cc9d6bb27 

Diff: https://reviews.apache.org/r/45392/diff/


Testing
---

Rendered version available at: 
https://github.com/StephanErb/aurora/tree/userguide_docu/docs


Thanks,

Stephan Erb

Re: Review Request 45193: Treat empty and null collections equivalently in task queries.

2016-03-28 Thread Bill Farner


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125678
---


Ship it!




- Bill Farner


On March 28, 2016, 8:33 a.m., John Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> ---
> 
> (Updated March 28, 2016, 8:33 a.m.)
> 
> 
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets).  In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
> 
>  src/main/java/org/apache/aurora/scheduler/base/JobKeys.java  
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/base/Query.java
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java  
> |  6 --
>  src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> |  6 +++---
>  src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> | 20 +---
>  6 files changed, 27 insertions(+), 11 deletions(-)
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 
> 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 
>   src/main/java/org/apache/aurora/scheduler/base/Query.java 
> ee01eaa4d0230d6bf0909b6460f27a74f03240db 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> ac0bb374842741d7ccb7a83c574a90ac156af0f9 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 
> 078dd8b63fdca192c735f9097edd030ee315a021 
>   src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 
> 4bf40047e105389ac7139edc449857889d390106 
>   src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 
> 231a55615abfbb483667f5f8ef71d2709fc16a88 
>   src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
> d326d24dd527d084bce1b300f1818d3b1d94c036 
>   
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java
>  5d246bee1a4dabc563a23c542384205537719f6a 
>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> 684614ffc42dd6778c7675a6c2f81cb72c106c0e 
>   
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java
>  dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java
>  3ba03429748448642571cfe0858278a50148745a 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
>  0a7b518578f4fd62c22e3ba52d8beae7958dc9eb 
> 
> Diff: https://reviews.apache.org/r/45193/diff/
> 
> 
> Testing
> ---
> 
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
> 
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
> 
> 
> Thanks,
> 
> John Sirois
> 
>

Re: Review Request 45212: Remove hard dependency on a specific mesos-version

2016-03-28 Thread Zameer Manji


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45212/#review125674
---


Ship it!




Ship It!

- Zameer Manji


On March 23, 2016, 7:56 a.m., Pierre Cheynier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45212/
> ---
> 
> (Updated March 23, 2016, 7:56 a.m.)
> 
> 
> Review request for Aurora, Jake Farrell, John Sirois, Stephan Erb, Bill 
> Farner, and Zameer Manji.
> 
> 
> Repository: aurora-packaging
> 
> 
> Description
> ---
> 
> We should consider MESOS_VERSION as the minimal requirement to install
> the current Aurora version instead of enforce a specific Mesos version.
> 
> 
> Diffs
> -
> 
>   specs/rpm/aurora.spec 61e7d146108ae7dd5e129d8288a05773c2659d25 
> 
> Diff: https://reviews.apache.org/r/45212/diff/
> 
> 
> Testing
> ---
> 
> Install Aurora through the RPM built with aurora-packaging on a Mesos 0.27
> running install.
> 
> 
> Thanks,
> 
> Pierre Cheynier
> 
>

Re: Review Request 45212: Remove hard dependency on a specific mesos-version

2016-03-28 Thread Maxim Khutornenko



> On March 28, 2016, 3:30 p.m., John Sirois wrote:
> > Zameer - this one is yours to take home.  I figure you have Maxim's ear and 
> > you two can work out the remainder and patch this in.

I still don't see much value in loosening version boundaries but don't intend 
to impede the consensus here.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45212/#review125642
---


On March 23, 2016, 2:56 p.m., Pierre Cheynier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45212/
> ---
> 
> (Updated March 23, 2016, 2:56 p.m.)
> 
> 
> Review request for Aurora, Jake Farrell, John Sirois, Stephan Erb, Bill 
> Farner, and Zameer Manji.
> 
> 
> Repository: aurora-packaging
> 
> 
> Description
> ---
> 
> We should consider MESOS_VERSION as the minimal requirement to install
> the current Aurora version instead of enforce a specific Mesos version.
> 
> 
> Diffs
> -
> 
>   specs/rpm/aurora.spec 61e7d146108ae7dd5e129d8288a05773c2659d25 
> 
> Diff: https://reviews.apache.org/r/45212/diff/
> 
> 
> Testing
> ---
> 
> Install Aurora through the RPM built with aurora-packaging on a Mesos 0.27
> running install.
> 
> 
> Thanks,
> 
> Pierre Cheynier
> 
>

Re: Review Request 45222: Introduce "preemptible" flag in TierInfo with backward compatible support for "production" flag in TaskConfig.

2016-03-28 Thread Maxim Khutornenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45222/#review125671
---


Ship it!





src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
 (line 130)


Any reason to allow more than "once" (default) here? Same in other tests.


- Maxim Khutornenko


On March 26, 2016, 12:08 a.m., Amol Deshmukh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45222/
> ---
> 
> (Updated March 26, 2016, 12:08 a.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Maxim Khutornenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1616
> https://issues.apache.org/jira/browse/AURORA-1616
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Introduce "preemptible" flag in TierInfo with backward compatible support for 
> "production" flag in TaskConfig.
> 
> # Summary of changes
> - TierInfo extended to manage a new property: "preemptible"
> - If TaskConfig does not specify a tier, ``TierManager.getTier(..)`` falls 
> back to selecting a non-revocable tier matching ``TaskConfig.isProduction()``.
> - Removed ``TierInfo.DEFAULT`` and replaced its in test/benchmark code by 
> references to ``TaskTestUtil.DEV_TIER``.
> - Eager validation of tier specified in TaskConfig in 
> ``ConfigurationManager.validateAndPopulate(..)``
> 
> # Note on backward incompatible change
> TaskConfigs with a tier name not matching any tiers defined in ``tiers.json`` 
> were silently assigned a default tier. With this change, attempting to 
> schedule tasks that specify non-existent tier names will result in an error 
> (see ``ConfigurationManager.validateAndPopulate(ITaskConfig config)``).
> 
> 
> Diffs
> -
> 
>   src/jmh/java/org/apache/aurora/benchmark/Offers.java 
> 055a2ffcb13c643a3086343e3fbf71545c5fb0a6 
>   src/main/java/org/apache/aurora/scheduler/TierInfo.java 
> b4611b9cf99ca236cd1ea4610d01c0d293bf2e87 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java 
> 480c4853a6c87dd63a9810ae013e5cfacb29336d 
>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java 
> 97d87ff1b789625f2c07baf7a74a076f07742596 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 
> 9cf8002e932d562e93fb8d17b4c56f564eb54ed5 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  b3b8ccf868c2a2f18f720a837e90d763072dd3eb 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> 16fa2d35cdf7ecdf82dd1ec7739e685ec1ff1aa2 
>   
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java
>  7f84e90774193b0d31adb7dafcab0a249167cdba 
>   src/main/resources/org/apache/aurora/scheduler/tiers.json 
> 18701058076bedc5d1b667e2b97ad09ce84a9bb9 
>   src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java 
> 39096af816864ada32a9c07958975740e805f6b0 
>   src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 
> 52113b80d91ecaf0cc2aeaad77e5fbc0ea4d1216 
>   src/test/java/org/apache/aurora/scheduler/ResourcesTest.java 
> 4fe8c518c580418078275b8056a5c195a765681e 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 
> a662100ba726cff0e47b4f9650753db9cecdef51 
>   src/test/java/org/apache/aurora/scheduler/TierModuleTest.java 
> e0601c83486671596e412f022ffae78b01c81c9d 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
>  1a520b3cb035a9afc25406d2f313c9f861eee4d6 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> 5103bd0f43d53079976b0f1596e299f2d91aa860 
>   
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
>  b6f5e4632ac1e028fdf93da1735463373e2d2788 
>   src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 
> b00add0b2fd4277e196505fffba4440e2e94207e 
>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 
> b1c71a6f1847d205c378d0b5a7269ea9d1165be5 
> 
> Diff: https://reviews.apache.org/r/45222/diff/
> 
> 
> Testing
> ---
> 
> # End to end tests
> ```
> $ ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ...
> *** OK (All tests passed) ***
> ```
> 
> # Jenkins build.sh
> ```
> $ ./build-support/jenkins/build.sh
> ...
>SUCCESS
> ```
> 
> # Local Scheduler
> ```
> $ ./gradlew run
> ...
> I0325 16:00:55.374 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> I0325 16:01:00.371 [pool-9-thread-1, FakeMaster:139] All offers consumed, 
> suppressing offer cycle.
> ```
> 
> # Attempting to schedule job with invalid tier-name
> ```
> vagrant@aurora:~/workspace$ aurora job create devcluster/vagrant/devel/test 
> job.aurora
>  INFO]

Re: Review Request 45372: Remove sleep and address flaky health check test.

2016-03-28 Thread John Sirois



> On March 28, 2016, 6:03 a.m., Stephan Erb wrote:
> > FWIW, there is als this very old review requests that is talking about the 
> > same tests https://reviews.apache.org/r/31380/diff/1#index_header. What 
> > does Brian mean with "calling .converge"?

He means the calls to this in the test code: 
https://github.com/twitter/commons/blob/master/src/python/twitter/common/testing/clock.py#L109
I'm also interested in that approach since the failures I saw from AuroraBot in 
https://reviews.apache.org/r/45366/ were both off by the epsilon.


- John


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45372/#review125628
---


On March 27, 2016, 9:41 p.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45372/
> ---
> 
> (Updated March 27, 2016, 9:41 p.m.)
> 
> 
> Review request for Aurora, John Sirois and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Posting this patch to at least start a conversation on fixing this test; i've 
> noticed it flaking pretty frequently lately.  Here i take the quick and dirty 
> approach of removing the sleep and glossing over the `total_latency_secs` 
> value.
> 
> 
> Diffs
> -
> 
>   src/test/python/apache/aurora/executor/common/test_health_checker.py 
> 19c4f76347e34374c29974c182d1f4c118bcb18d 
> 
> Diff: https://reviews.apache.org/r/45372/diff/
> 
> 
> Testing
> ---
> 
> None yet
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

Re: Review Request 45193: Treat empty and null collections equivalently in task queries.

2016-03-28 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125657
---


Ship it!




Master (83a078b) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 28, 2016, 3:33 p.m., John Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> ---
> 
> (Updated March 28, 2016, 3:33 p.m.)
> 
> 
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets).  In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
> 
>  src/main/java/org/apache/aurora/scheduler/base/JobKeys.java  
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/base/Query.java
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java  
> |  6 --
>  src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> |  6 +++---
>  src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> | 20 +---
>  6 files changed, 27 insertions(+), 11 deletions(-)
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 
> 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 
>   src/main/java/org/apache/aurora/scheduler/base/Query.java 
> ee01eaa4d0230d6bf0909b6460f27a74f03240db 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> ac0bb374842741d7ccb7a83c574a90ac156af0f9 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 
> 078dd8b63fdca192c735f9097edd030ee315a021 
>   src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 
> 4bf40047e105389ac7139edc449857889d390106 
>   src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 
> 231a55615abfbb483667f5f8ef71d2709fc16a88 
>   src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
> d326d24dd527d084bce1b300f1818d3b1d94c036 
>   
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java
>  5d246bee1a4dabc563a23c542384205537719f6a 
>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> 684614ffc42dd6778c7675a6c2f81cb72c106c0e 
>   
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java
>  dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java
>  3ba03429748448642571cfe0858278a50148745a 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
>  0a7b518578f4fd62c22e3ba52d8beae7958dc9eb 
> 
> Diff: https://reviews.apache.org/r/45193/diff/
> 
> 
> Testing
> ---
> 
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
> 
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
> 
> 
> Thanks,
> 
> John Sirois
> 
>

Re: Review Request 45193: Treat empty and null collections equivalently in task queries.

2016-03-28 Thread John Sirois


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125644
---



Bill - I'll take silence as consent and merge this today.

- John Sirois


On March 28, 2016, 9:33 a.m., John Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> ---
> 
> (Updated March 28, 2016, 9:33 a.m.)
> 
> 
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets).  In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
> 
>  src/main/java/org/apache/aurora/scheduler/base/JobKeys.java  
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/base/Query.java
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java  
> |  6 --
>  src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> |  6 +++---
>  src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> | 20 +---
>  6 files changed, 27 insertions(+), 11 deletions(-)
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 
> 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 
>   src/main/java/org/apache/aurora/scheduler/base/Query.java 
> ee01eaa4d0230d6bf0909b6460f27a74f03240db 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> ac0bb374842741d7ccb7a83c574a90ac156af0f9 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 
> 078dd8b63fdca192c735f9097edd030ee315a021 
>   src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 
> 4bf40047e105389ac7139edc449857889d390106 
>   src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 
> 231a55615abfbb483667f5f8ef71d2709fc16a88 
>   src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
> d326d24dd527d084bce1b300f1818d3b1d94c036 
>   
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java
>  5d246bee1a4dabc563a23c542384205537719f6a 
>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> 684614ffc42dd6778c7675a6c2f81cb72c106c0e 
>   
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java
>  dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java
>  3ba03429748448642571cfe0858278a50148745a 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
>  0a7b518578f4fd62c22e3ba52d8beae7958dc9eb 
> 
> Diff: https://reviews.apache.org/r/45193/diff/
> 
> 
> Testing
> ---
> 
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
> 
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
> 
> 
> Thanks,
> 
> John Sirois
> 
>

Re: Review Request 45193: Treat empty and null collections equivalently in task queries.

2016-03-28 Thread John Sirois


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/
---

(Updated March 28, 2016, 9:33 a.m.)


Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.


Changes
---

Back task queries with `ITaskQuery`.

This avoids special null checks for collection query criteria although
it does require more care in struct handling code where `TaskQuery` <->
`ITaskQuery` are lossy for `isSet` query methods.

 src/main/java/org/apache/aurora/scheduler/base/JobKeys.java
|  8 +++-
 src/main/java/org/apache/aurora/scheduler/base/Query.java  
| 19 +--
 src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java   
| 15 +++
 src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java  
|  2 +-
 src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java   
|  4 ++--
 src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java
| 14 +++---
 src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java
| 12 
 src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 
|  2 +-
 src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml   
| 10 +-
 
src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java 
   | 18 ++
 
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 
   |  6 +++---
 
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
 | 20 ++--
 12 files changed, 66 insertions(+), 64 deletions(-)


Repository: aurora


Description
---

Previously, `null` was handled differently from an empty collection in
task queries. For the Go thrift bindings, this was problematic since
zero values in Go are useful in almost all cases and in particular in the
case of maps (used to represent sets).  In these cases unset `TaskQuery`
collection parameters are serialized as empty collections (empty
maps) instead of `nil` (`null`), leading to the inability to use the
query API in any natural way.

 src/main/java/org/apache/aurora/scheduler/base/JobKeys.java  | 
 2 +-
 src/main/java/org/apache/aurora/scheduler/base/Query.java| 
 2 +-
 src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 
 2 +-
 src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java  | 
 6 --
 src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 
 6 +++---
 src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 
20 +---
 6 files changed, 27 insertions(+), 11 deletions(-)


Diffs (updated)
-

  src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 
8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 
  src/main/java/org/apache/aurora/scheduler/base/Query.java 
ee01eaa4d0230d6bf0909b6460f27a74f03240db 
  src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
ac0bb374842741d7ccb7a83c574a90ac156af0f9 
  src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 
078dd8b63fdca192c735f9097edd030ee315a021 
  src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 
4bf40047e105389ac7139edc449857889d390106 
  src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 
231a55615abfbb483667f5f8ef71d2709fc16a88 
  src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
d326d24dd527d084bce1b300f1818d3b1d94c036 
  
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 
5d246bee1a4dabc563a23c542384205537719f6a 
  src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
684614ffc42dd6778c7675a6c2f81cb72c106c0e 
  
src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java 
dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 
  src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 
  
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 
3ba03429748448642571cfe0858278a50148745a 
  
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb 

Diff: https://reviews.apache.org/r/45193/diff/


Testing
---

NB: This change was broken out of https://reviews.apache.org/r/42756/
since it stands on its own (although its slightly more awkward in the
mutable thrift world) and the case of the Go Aurora API client forces the
issue.

Locally green:
```
./build-support/jenkins/build.sh
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
```


Thanks,

John Sirois

Re: Review Request 45193: Treat empty and null collections equivalently in task queries.

2016-03-28 Thread John Sirois


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125643
---



Rebasing to pick up https://reviews.apache.org/r/45366/

- John Sirois


On March 24, 2016, 8:52 a.m., John Sirois wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> ---
> 
> (Updated March 24, 2016, 8:52 a.m.)
> 
> 
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets).  In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
> 
>  src/main/java/org/apache/aurora/scheduler/base/JobKeys.java  
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/base/Query.java
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> |  2 +-
>  src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java  
> |  6 --
>  src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> |  6 +++---
>  src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> | 20 +---
>  6 files changed, 27 insertions(+), 11 deletions(-)
> 
> 
> Diffs
> -
> 
>   src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 
> 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242 
>   src/main/java/org/apache/aurora/scheduler/base/Query.java 
> ee01eaa4d0230d6bf0909b6460f27a74f03240db 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 
> ac0bb374842741d7ccb7a83c574a90ac156af0f9 
>   src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 
> 078dd8b63fdca192c735f9097edd030ee315a021 
>   src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 
> 4bf40047e105389ac7139edc449857889d390106 
>   src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 
> 231a55615abfbb483667f5f8ef71d2709fc16a88 
>   src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 
> d326d24dd527d084bce1b300f1818d3b1d94c036 
>   
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java
>  5d246bee1a4dabc563a23c542384205537719f6a 
>   src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 
> 684614ffc42dd6778c7675a6c2f81cb72c106c0e 
>   
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java
>  dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java
>  3ba03429748448642571cfe0858278a50148745a 
>   
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
>  0a7b518578f4fd62c22e3ba52d8beae7958dc9eb 
> 
> Diff: https://reviews.apache.org/r/45193/diff/
> 
> 
> Testing
> ---
> 
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
> 
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
> 
> 
> Thanks,
> 
> John Sirois
> 
>

Re: Review Request 45212: Remove hard dependency on a specific mesos-version

2016-03-28 Thread John Sirois


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45212/#review125642
---



Zameer - this one is yours to take home.  I figure you have Maxim's ear and you 
two can work out the remainder and patch this in.

- John Sirois


On March 23, 2016, 8:56 a.m., Pierre Cheynier wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45212/
> ---
> 
> (Updated March 23, 2016, 8:56 a.m.)
> 
> 
> Review request for Aurora, Jake Farrell, John Sirois, Stephan Erb, Bill 
> Farner, and Zameer Manji.
> 
> 
> Repository: aurora-packaging
> 
> 
> Description
> ---
> 
> We should consider MESOS_VERSION as the minimal requirement to install
> the current Aurora version instead of enforce a specific Mesos version.
> 
> 
> Diffs
> -
> 
>   specs/rpm/aurora.spec 61e7d146108ae7dd5e129d8288a05773c2659d25 
> 
> Diff: https://reviews.apache.org/r/45212/diff/
> 
> 
> Testing
> ---
> 
> Install Aurora through the RPM built with aurora-packaging on a Mesos 0.27
> running install.
> 
> 
> Thanks,
> 
> Pierre Cheynier
> 
>

Re: Review Request 45372: Remove sleep and address flaky health check test.

2016-03-28 Thread Stephan Erb


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45372/#review125628
---



FWIW, there is als this very old review requests that is talking about the same 
tests https://reviews.apache.org/r/31380/diff/1#index_header. What does Brian 
mean with "calling .converge"?

- Stephan Erb


On March 28, 2016, 5:41 a.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45372/
> ---
> 
> (Updated March 28, 2016, 5:41 a.m.)
> 
> 
> Review request for Aurora, John Sirois and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Posting this patch to at least start a conversation on fixing this test; i've 
> noticed it flaking pretty frequently lately.  Here i take the quick and dirty 
> approach of removing the sleep and glossing over the `total_latency_secs` 
> value.
> 
> 
> Diffs
> -
> 
>   src/test/python/apache/aurora/executor/common/test_health_checker.py 
> 19c4f76347e34374c29974c182d1f4c118bcb18d 
> 
> Diff: https://reviews.apache.org/r/45372/diff/
> 
> 
> Testing
> ---
> 
> None yet
> 
> 
> Thanks,
> 
> Bill Farner
> 
>

41 matches

Mail list logo