Re: Review Request 44871: Speed up Vagrant provisioning by using a custom base box.

2016-03-19 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44871/
---

(Updated March 16, 2016, 8:51 p.m.)


Review request for Aurora, John Sirois and Zameer Manji.


Changes
---

Addressed startup issues that jsirois and i encountered.  This was an ordering 
issue, where init scripts from the mesos package were being used rather than 
the upstart scripts in our repo.  In the latest revision of the patch, i 
dropped our upstart config and instead configure the package we are consume.  

Unfortunately there's an issue i encountered down the road related to 
docker/devicemapper.  This should not stall review, however.


Repository: aurora


Description (updated)
---

_READY FOR REVIEW BUT NOT SUBMISSION - END-TO-END TESTS FAIL_

Relevant docs on packer can be found here: https://www.packer.io/docs/

To support this, i have created the [`apache-aurora` 
org](https://atlas.hashicorp.com/apache-aurora).  If we move forward with this 
patch, i will be looking to add other committers to the org and improve the bus 
factor.


Diffs (updated)
-

  .gitignore 1af09a251b3f76c13813033d32aa7efba9aef304 
  Vagrantfile 2d6c2ae598e80035840f7e517e161be266b581dd 
  build-support/packer/README.md PRE-CREATION 
  build-support/packer/aurora.json PRE-CREATION 
  build-support/packer/build.sh PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-master/ip PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-master/roles PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/containerizers PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/hostname PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/ip PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/modules PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/resource_estimator PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/resources PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/work_dir PRE-CREATION 
  examples/vagrant/provision-dev-cluster.sh 
df33a30c1dcc0ea832899c849be05fe9deb47433 
  examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
eec5ee3a2f041ae25fa664b80004e27ed8b5e2b0 
  examples/vagrant/upstart/aurora-scheduler.conf 
7e10b2c7690087bb150a9033cbd576ab82b1b185 
  examples/vagrant/upstart/mesos-master.conf 
a2ed953d871a7b28444a188ee0fc152853c5b088 
  examples/vagrant/upstart/mesos-slave.conf 
d97fa1fcf99580ef3f0997e9c78cb9775b91a818 

Diff: https://reviews.apache.org/r/44871/diff/


Testing (updated)
---

I can run `vagrant up`, end-to-end tests pass.

Reviewers - you can do this as well by applying the patch.  You will observe a 
long fetch on the first run (much like you would with the current repo if you 
wiped your local vagrant boxes).  However, subsequent `vagrant destroy`, 
`vagrant up` cycles should be relatively fast, about 1m30s for me.


Thanks,

Bill Farner

Re: Review Request 45042: Add ACL support for announcer

2016-03-19 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review124431
---



Overall looks in decent shape, made some suggestions.

Before this is ready to land, please:
- add documentation
- cross-link with the jira ticket


src/main/python/apache/aurora/executor/common/announcer.py (line 128)


please refactor to make the constructor accept the ACL list, making the 
caller read/parse.  this improves testability and reduces the surface area of 
the class.  it also moves errors like file access and parsing issues closer to 
the source



src/main/python/apache/aurora/executor/common/announcer.py (lines 149 - 151)


please read the json as a list, to support multiple ACLs (mentioned this in 
the ticket)



src/main/python/apache/aurora/executor/common/announcer.py (line 153)


-1 on the approach of logging and moving forward without auth when errors 
are encountered.  these should all be fatal



src/main/python/apache/aurora/executor/common/announcer.py (line 162)


General tip, avoid relying on truthiness when you really mean to compare 
against `None`

From https://www.python.org/dev/peps/pep-0008/#programming-recommendations

> beware of writing if x when you really mean if x is not None -- e.g. when 
testing whether a variable or argument that defaults to None was set to some 
other value. The other value might have a type (such as a container) that could 
be false in a boolean context


- Bill Farner


On March 18, 2016, 2:57 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 18, 2016, 2:57 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> ```json
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
> ```
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 45076: Improve presentation and organization of the scheduler landing page

2016-03-19 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45076/#review124424
---


Ship it!




Ship It!

- Bill Farner


On March 19, 2016, 1 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45076/
> ---
> 
> (Updated March 19, 2016, 1 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> I have opted for a very simple, incremental improvement here.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/index.html 
> d0bd897524a4c763d11a601898b663fec672e979 
> 
> Diff: https://reviews.apache.org/r/45076/diff/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> aurora-landing-page.png
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/19/9a809739-966a-49e3-bf69-df03754be1c8__aurora-landing-page.png
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 45076: Improve presentation and organization of the scheduler landing page

2016-03-19 Thread Bill Farner 


> On March 19, 2016, 1:44 p.m., Aurora ReviewBot wrote:
> > Master (b24619b) is red with this patch.
> >   ./build-support/jenkins/build.sh
> > 
> > CAP#1 extends Unit from capture of ?
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/args/parsers/ListParser.java:44:
> >  error: incompatible types: cannot infer type-variable(s) E,F,T
> > return 
> > ImmutableList.copyOf(Iterables.transform(Parsers.MULTI_VALUE_SPLITTER.split(raw),
> >^
> > (argument mismatch; Iterable cannot be converted to Iterable > extends CAP#1>)
> >   where E,F,T are type-variables:
> > E extends Object declared in method copyOf(Iterable)
> > F extends Object declared in method 
> > transform(Iterable,Function)
> > T extends Object declared in method 
> > transform(Iterable,Function)
> >   where CAP#1 is a fresh type-variable:
> > CAP#1 extends Object from capture of ?
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/args/parsers/SetParser.java:46:
> >  error: incompatible types: cannot infer type-variable(s) E,F,T
> > return 
> > ImmutableSet.copyOf(Iterables.transform(Parsers.MULTI_VALUE_SPLITTER.split(raw),
> >   ^
> > (argument mismatch; Iterable cannot be converted to Iterable > extends CAP#1>)
> >   where E,F,T are type-variables:
> > E extends Object declared in method copyOf(Iterable)
> > F extends Object declared in method 
> > transform(Iterable,Function)
> > T extends Object declared in method 
> > transform(Iterable,Function)
> >   where CAP#1 is a fresh type-variable:
> > CAP#1 extends Object from capture of ?
> > Note: 
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java
> >  uses or overrides a deprecated API.
> > Note: Recompile with -Xlint:deprecation for details.
> > Note: Some input files use unchecked or unsafe operations.
> > Note: Recompile with -Xlint:unchecked for details.
> > 8 errors
> >  FAILED
> > 
> > FAILURE: Build failed with an exception.
> > 
> > * What went wrong:
> > Execution failed for task ':commons:compileJava'.
> > > Compilation failed; see the compiler error output for details.
> > 
> > * Try:
> > Run with --stacktrace option to get the stack trace. Run with --info or 
> > --debug option to get more log output.
> > 
> > BUILD FAILED
> > 
> > Total time: 2 mins 36.803 secs
> > 
> > 
> > I will refresh this build result if you post a review containing 
> > "@ReviewBot retry"

Weird, let's ignore the build result for the purposes of this review - it's 
clearly false.  I'll poke around in jenkins, wonder if the JDK version changed 
on us.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45076/#review124421
---


On March 19, 2016, 1 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45076/
> ---
> 
> (Updated March 19, 2016, 1 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> I have opted for a very simple, incremental improvement here.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/index.html 
> d0bd897524a4c763d11a601898b663fec672e979 
> 
> Diff: https://reviews.apache.org/r/45076/diff/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> aurora-landing-page.png
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/19/9a809739-966a-49e3-bf69-df03754be1c8__aurora-landing-page.png
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 45076: Improve presentation and organization of the scheduler landing page

2016-03-19 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45076/#review124421
---



Master (b24619b) is red with this patch.
  ./build-support/jenkins/build.sh

CAP#1 extends Unit from capture of ?
/home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/args/parsers/ListParser.java:44:
 error: incompatible types: cannot infer type-variable(s) E,F,T
return 
ImmutableList.copyOf(Iterables.transform(Parsers.MULTI_VALUE_SPLITTER.split(raw),
   ^
(argument mismatch; Iterable cannot be converted to Iterable)
  where E,F,T are type-variables:
E extends Object declared in method copyOf(Iterable)
F extends Object declared in method transform(Iterable,Function)
T extends Object declared in method transform(Iterable,Function)
  where CAP#1 is a fresh type-variable:
CAP#1 extends Object from capture of ?
/home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/args/parsers/SetParser.java:46:
 error: incompatible types: cannot infer type-variable(s) E,F,T
return 
ImmutableSet.copyOf(Iterables.transform(Parsers.MULTI_VALUE_SPLITTER.split(raw),
  ^
(argument mismatch; Iterable cannot be converted to Iterable)
  where E,F,T are type-variables:
E extends Object declared in method copyOf(Iterable)
F extends Object declared in method transform(Iterable,Function)
T extends Object declared in method transform(Iterable,Function)
  where CAP#1 is a fresh type-variable:
CAP#1 extends Object from capture of ?
Note: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java
 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
8 errors
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':commons:compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

BUILD FAILED

Total time: 2 mins 36.803 secs


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 19, 2016, 8 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45076/
> ---
> 
> (Updated March 19, 2016, 8 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> I have opted for a very simple, incremental improvement here.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/index.html 
> d0bd897524a4c763d11a601898b663fec672e979 
> 
> Diff: https://reviews.apache.org/r/45076/diff/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> aurora-landing-page.png
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/19/9a809739-966a-49e3-bf69-df03754be1c8__aurora-landing-page.png
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 44871: Speed up Vagrant provisioning by using a custom base box.

2016-03-19 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44871/
---

(Updated March 17, 2016, 2:21 p.m.)


Review request for Aurora, John Sirois and Zameer Manji.


Changes
---

Removed some large/unused packages to trim down the base box size.


Repository: aurora


Description
---

Relevant docs on packer can be found here: https://www.packer.io/docs/

To support this, i have created the [`apache-aurora` 
org](https://atlas.hashicorp.com/apache-aurora).  If we move forward with this 
patch, i will be looking to add other committers to the org and improve the bus 
factor.


Diffs (updated)
-

  .gitignore 1af09a251b3f76c13813033d32aa7efba9aef304 
  Vagrantfile 2d6c2ae598e80035840f7e517e161be266b581dd 
  build-support/packer/README.md PRE-CREATION 
  build-support/packer/aurora.json PRE-CREATION 
  build-support/packer/build.sh PRE-CREATION 
  examples/vagrant/mesos_config/README.md PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-master/ip PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-master/roles PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/containerizers PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/hostname PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/ip PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/modules PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/resource_estimator PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/resources PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/work_dir PRE-CREATION 
  examples/vagrant/provision-dev-cluster.sh 
df33a30c1dcc0ea832899c849be05fe9deb47433 
  examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
eec5ee3a2f041ae25fa664b80004e27ed8b5e2b0 
  examples/vagrant/upstart/aurora-scheduler.conf 
7e10b2c7690087bb150a9033cbd576ab82b1b185 
  examples/vagrant/upstart/mesos-master.conf 
a2ed953d871a7b28444a188ee0fc152853c5b088 
  examples/vagrant/upstart/mesos-slave.conf 
d97fa1fcf99580ef3f0997e9c78cb9775b91a818 

Diff: https://reviews.apache.org/r/44871/diff/


Testing
---

I can run `vagrant up`, end-to-end tests pass.

Reviewers - you can do this as well by applying the patch.  You will observe a 
long fetch on the first run (much like you would with the current repo if you 
wiped your local vagrant boxes).  However, subsequent `vagrant destroy`, 
`vagrant up` cycles should be relatively fast, about 1m30s for me.


Thanks,

Bill Farner

Re: Review Request 45076: Improve presentation and organization of the scheduler landing page

2016-03-19 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45076/#review124418
---



Master (b24619b) is red with this patch.
  ./build-support/jenkins/build.sh

CAP#1 extends Unit from capture of ?
/home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/args/parsers/ListParser.java:44:
 error: incompatible types: cannot infer type-variable(s) E,F,T
return 
ImmutableList.copyOf(Iterables.transform(Parsers.MULTI_VALUE_SPLITTER.split(raw),
   ^
(argument mismatch; Iterable cannot be converted to Iterable)
  where E,F,T are type-variables:
E extends Object declared in method copyOf(Iterable)
F extends Object declared in method transform(Iterable,Function)
T extends Object declared in method transform(Iterable,Function)
  where CAP#1 is a fresh type-variable:
CAP#1 extends Object from capture of ?
/home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/args/parsers/SetParser.java:46:
 error: incompatible types: cannot infer type-variable(s) E,F,T
return 
ImmutableSet.copyOf(Iterables.transform(Parsers.MULTI_VALUE_SPLITTER.split(raw),
  ^
(argument mismatch; Iterable cannot be converted to Iterable)
  where E,F,T are type-variables:
E extends Object declared in method copyOf(Iterable)
F extends Object declared in method transform(Iterable,Function)
T extends Object declared in method transform(Iterable,Function)
  where CAP#1 is a fresh type-variable:
CAP#1 extends Object from capture of ?
Note: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java
 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
8 errors
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':commons:compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

BUILD FAILED

Total time: 7 mins 17.902 secs


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 19, 2016, 8 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45076/
> ---
> 
> (Updated March 19, 2016, 8 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> I have opted for a very simple, incremental improvement here.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/index.html 
> d0bd897524a4c763d11a601898b663fec672e979 
> 
> Diff: https://reviews.apache.org/r/45076/diff/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> aurora-landing-page.png
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/19/9a809739-966a-49e3-bf69-df03754be1c8__aurora-landing-page.png
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 45076: Improve presentation and organization of the scheduler landing page

2016-03-19 Thread Stephan Erb 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45076/
---

(Updated March 19, 2016, 9 p.m.)


Review request for Aurora, Joshua Cohen and Bill Farner.


Repository: aurora


Description
---

I have opted for a very simple, incremental improvement here.


Diffs
-

  src/main/resources/scheduler/assets/index.html 
d0bd897524a4c763d11a601898b663fec672e979 

Diff: https://reviews.apache.org/r/45076/diff/


Testing
---


File Attachments (updated)


aurora-landing-page.png
  
https://reviews.apache.org/media/uploaded/files/2016/03/19/9a809739-966a-49e3-bf69-df03754be1c8__aurora-landing-page.png


Thanks,

Stephan Erb

Review Request 45076: Improve presentation and organization of the scheduler landing page

2016-03-19 Thread Stephan Erb 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45076/
---

Review request for Aurora, Joshua Cohen and Bill Farner.


Repository: aurora


Description
---

I have opted for a very simple, incremental improvement here.


Diffs
-

  src/main/resources/scheduler/assets/index.html 
d0bd897524a4c763d11a601898b663fec672e979 

Diff: https://reviews.apache.org/r/45076/diff/


Testing
---


Thanks,

Stephan Erb

Re: Review Request 44532: Allow overriding hostname before announcing

2016-03-19 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44532/#review123965
---


Fix it, then Ship it!




Just one nit from me, otherwise LGTM!


src/main/python/apache/aurora/executor/common/announcer.py (line 60)


This pre-dates your change in this file, but the double underscore for 
'private' fields doesn't match the convention in the rest of the project.  
Please change to single underscore (same for 
`self.__allow_custom_serverset_path`).


- Bill Farner


On March 16, 2016, 12:43 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44532/
> ---
> 
> (Updated March 16, 2016, 12:43 p.m.)
> 
> 
> Review request for Aurora, John Sirois, Stephan Erb, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Allow overriding hostname before announcing 
> (https://issues.apache.org/jira/browse/AURORA-1611)
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> f82858c528808d2a9e77bb56f16e897cfb5bbe73 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 34e36e0a59093468a8934f58bacb68512949347c 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> f4032c7302f4733ab5670322b1905102c200f1c9 
> 
> Diff: https://reviews.apache.org/r/44532/diff/
> 
> 
> Testing
> ---
> 
> Tested on vagrant with a wrapper for the executor
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 45042: Add ACL support for announcer

2016-03-19 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/#review124334
---



Master (b24619b) is red with this patch.
  ./build-support/jenkins/build.sh

   proxy_driver = ProxyDriver()
   with temporary_dir() as checkpoint_root:
 te = AuroraExecutor(
 >   
runner_provider=make_provider(checkpoint_root),
 
sandbox_provider=DefaultTestSandboxProvider())
 
 
src/test/python/apache/aurora/executor/test_thermos_executor.py:580: 
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 
src/test/python/apache/aurora/executor/test_thermos_executor.py:193: in 
make_provider
 pex_location=thermos_runner_path(),
 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 
 build = True
 
 def thermos_runner_path(build=True):
   if not build:
 return getattr(thermos_runner_path, 'value', 
None)
 
   if not hasattr(thermos_runner_path, 'value'):
 pex_dir = safe_mkdtemp()
 >   assert subprocess.call(["./pants", 
"--pants-distdir=%s" % pex_dir, "binary",
   
"src/main/python/apache/thermos/runner:thermos_runner"]) == 0
 E   assert 1 == 0
 E+  where 1 = (['./pants', '--pants-distdir=/tmp/user/10021/tmpXl2PW0', 
'binary', 'src/main/python/apache/thermos/runner:thermos_runner'])
 E+where  = subprocess.call
 
 
src/test/python/apache/aurora/executor/test_thermos_executor.py:185: 
AssertionError
 -- Captured stderr call --
 Traceback (most recent call last):
   File 
"/home/jenkins/.cache/pants/setup/bootstrap-Linux-x86_64/0.0.75/bin/pants", 
line 7, in 
 from pants.bin.pants_exe import main
 ImportError: No module named pants.bin.pants_exe
  generated xml file: 
/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/415337499eb72578eab327a6487c1f5c9452b3d6.xml
 
  16 failed, 643 passed, 5 skipped, 1 warnings, 8 
error in 228.64 seconds 
 
FAILURE


00:17:28 04:44   [complete]
   FAILURE


I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 18, 2016, 9:57 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45042/
> ---
> 
> (Updated March 18, 2016, 9:57 p.m.)
> 
> 
> Review request for Aurora, Bill Farner and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add ACL support for announcer
> https://issues.apache.org/jira/browse/AURORA-1643
> 
> Adding support for service discovery ZK authentication. ZK authentication 
> secrets should be stored in a file as json (as follows):
> ```json
> {
>   "scheme": "",
>   "credential": "",
>   "permissions": {
> "read": ,
> "write": ,
> "create": ,
> "delete": ,
> "admin": ,
> "all": 
>   }
> }
> ```
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> 6634506108c346f8c23b2da7cc8d20d09d07d590 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 79a9cfb6ac3a8444f09fb3658e6e859e06941ba4 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> 142b58d5e577c9f4b8e2ae8473cffdea94eba21f 
> 
> Diff: https://reviews.apache.org/r/45042/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Review Request 45042: Add ACL support for announcer

2016-03-19 Thread Kunal Thakar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45042/
---

Review request for Aurora, Bill Farner and Zameer Manji.


Repository: aurora


Description
---

Add ACL support for announcer
https://issues.apache.org/jira/browse/AURORA-1643

Adding support for service discovery ZK authentication. ZK authentication 
secrets should be stored in a file as json (as follows):
```json
{
  "scheme": "",
  "credential": "",
"permissions": {
  "read": ,
  "write": ,
  "create": ,
  "delete": ,
  "admin": ,
  "all": 
}
}
```


Diffs
-

  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f82858c528808d2a9e77bb56f16e897cfb5bbe73 
  src/main/python/apache/aurora/executor/common/announcer.py 
34e36e0a59093468a8934f58bacb68512949347c 
  src/test/python/apache/aurora/executor/common/test_announcer.py 
f4032c7302f4733ab5670322b1905102c200f1c9 

Diff: https://reviews.apache.org/r/45042/diff/


Testing
---


Thanks,

Kunal Thakar

Re: Review Request 44770: Create scheduler-configuration.md

2016-03-19 Thread Stephan Erb 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44770/
---

(Updated March 18, 2016, 10:19 p.m.)


Review request for Aurora and Bill Farner.


Changes
---

Wrap at 100 cols.


Repository: aurora


Description
---

Having a simple dump of the all scheduler configuration options makes it much 
easier to discover or find them via a public search engine.

Ideally we would have a way to update this automatically, but anything is 
better than not having them listed at all.


Diffs (updated)
-

  docs/README.md 78f062abb86faf9ac0347c02828d78a544c74198 
  docs/deploying-aurora-scheduler.md 03ee360a398821dcc8ca74001bfac478241ad5d8 
  docs/scheduler-configuration.md PRE-CREATION 

Diff: https://reviews.apache.org/r/44770/diff/


Testing
---

Rendered version online at 
https://github.com/StephanErb/aurora/blob/cmdopts/docs/scheduler-configuration.md


Thanks,

Stephan Erb

Re: Review Request 44871: Speed up Vagrant provisioning by using a custom base box.

2016-03-19 Thread Bill Farner 


> On March 17, 2016, 11:01 a.m., Aurora ReviewBot wrote:
> > Master (6537581) is red with this patch.
> >   ./build-support/jenkins/build.sh
> > 
> >    proxy_driver = ProxyDriver()
> >    with temporary_dir() as checkpoint_root:
> >  te = AuroraExecutor(
> >  >   
> > runner_provider=make_provider(checkpoint_root),
> >  
> > sandbox_provider=DefaultTestSandboxProvider())
> >  
> >  
> > src/test/python/apache/aurora/executor/test_thermos_executor.py:580: 
> >  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> >  
> > src/test/python/apache/aurora/executor/test_thermos_executor.py:193: in 
> > make_provider
> >  pex_location=thermos_runner_path(),
> >  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> >  
> >  build = True
> >  
> >  def thermos_runner_path(build=True):
> >    if not build:
> >  return getattr(thermos_runner_path, 
> > 'value', None)
> >  
> >    if not hasattr(thermos_runner_path, 
> > 'value'):
> >  pex_dir = safe_mkdtemp()
> >  >   assert subprocess.call(["./pants", 
> > "--pants-distdir=%s" % pex_dir, "binary",
> >    
> > "src/main/python/apache/thermos/runner:thermos_runner"]) == 0
> >  E   assert 1 == 0
> >  E+  where 1 =  > 0x7f83cd239b18>(['./pants', '--pants-distdir=/tmp/user/10021/tmp0RrjAm', 
> > 'binary', 'src/main/python/apache/thermos/runner:thermos_runner'])
> >  E+where  > 0x7f83cd239b18> = subprocess.call
> >  
> >  
> > src/test/python/apache/aurora/executor/test_thermos_executor.py:185: 
> > AssertionError
> >  -- Captured stderr call --
> >  Traceback (most recent call last):
> >File 
> > "/home/jenkins/.cache/pants/setup/bootstrap-Linux-x86_64/0.0.75/bin/pants", 
> > line 7, in 
> >  from pants.bin.pants_exe import main
> >  ImportError: No module named pants.bin.pants_exe
> >   generated xml file: 
> > /home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/415337499eb72578eab327a6487c1f5c9452b3d6.xml
> >  
> >   16 failed, 639 passed, 5 skipped, 1 
> > warnings, 8 error in 216.63 seconds 
> >  
> > FAILURE
> > 
> > 
> > 18:00:57 04:20   [complete]
> >FAILURE
> > 
> > 
> > I will refresh this build result if you post a review containing 
> > "@ReviewBot retry"

Noticed this error in another review reply.  I've wiped the workspace for this 
job in the hopes that it's an issue of bad state.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44871/#review124062
---


On March 17, 2016, 10:28 a.m., Bill Farner wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44871/
> ---
> 
> (Updated March 17, 2016, 10:28 a.m.)
> 
> 
> Review request for Aurora, John Sirois and Zameer Manji.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Relevant docs on packer can be found here: https://www.packer.io/docs/
> 
> To support this, i have created the [`apache-aurora` 
> org](https://atlas.hashicorp.com/apache-aurora).  If we move forward with 
> this patch, i will be looking to add other committers to the org and improve 
> the bus factor.
> 
> 
> Diffs
> -
> 
>   .gitignore 1af09a251b3f76c13813033d32aa7efba9aef304 
>   Vagrantfile 2d6c2ae598e80035840f7e517e161be266b581dd 
>   build-support/packer/README.md PRE-CREATION 
>   build-support/packer/aurora.json PRE-CREATION 
>   build-support/packer/build.sh PRE-CREATION 
>   examples/vagrant/mesos_config/README.md PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-master/ip PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-master/roles PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-slave/containerizers PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-slave/hostname PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-slave/ip

Re: Review Request 45055: Fix flickering of the Aurora logo on refresh.

2016-03-19 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45055/#review124331
---



Master (b24619b) is green with this patch.
  ./build-support/jenkins/build.sh

However, it appears that it might lack test coverage.

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 18, 2016, 10:14 p.m., Stephan Erb wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45055/
> ---
> 
> (Updated March 18, 2016, 10:14 p.m.)
> 
> 
> Review request for Aurora and Joshua Cohen.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Refreshing a subpage of /scheduler in Chrome temporarily resize the Aurora
> logo. This seems to be due to us loading all scripts and styles in the 
> after first rending the logo.
> 
> By loading all scripts in the header using the `defer` attribute, scripts are
> only executed after the entire document has been loaded. They are executed in
> order.
> 
> 
> Diffs
> -
> 
>   src/main/resources/scheduler/assets/scheduler/index.html 
> 383ca43dfc14448b67181e9d94dd805ac1bbfaa7 
> 
> Diff: https://reviews.apache.org/r/45055/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Stephan Erb
> 
>

Re: Review Request 44532: Allow overriding hostname before announcing

2016-03-19 Thread Kunal Thakar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44532/
---

(Updated March 16, 2016, 7:43 p.m.)


Review request for Aurora, John Sirois, Stephan Erb, and Bill Farner.


Repository: aurora


Description
---

Allow overriding hostname before announcing 
(https://issues.apache.org/jira/browse/AURORA-1611)


Diffs (updated)
-

  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f82858c528808d2a9e77bb56f16e897cfb5bbe73 
  src/main/python/apache/aurora/executor/common/announcer.py 
34e36e0a59093468a8934f58bacb68512949347c 
  src/test/python/apache/aurora/executor/common/test_announcer.py 
f4032c7302f4733ab5670322b1905102c200f1c9 

Diff: https://reviews.apache.org/r/44532/diff/


Testing
---

Tested on vagrant with a wrapper for the executor


Thanks,

Kunal Thakar

Re: Review Request 44532: Allow overriding hostname before announcing

2016-03-19 Thread Kunal Thakar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44532/
---

(Updated March 18, 2016, 7:04 p.m.)


Review request for Aurora, John Sirois, Stephan Erb, and Bill Farner.


Repository: aurora


Description
---

Allow overriding hostname before announcing 
(https://issues.apache.org/jira/browse/AURORA-1611)


Diffs (updated)
-

  RELEASE-NOTES.md 42bb340c492f25286c8c785885132abcbe455296 
  docs/configuration-reference.md e043c4886c6d0fd96646f4687865d1c715cd1c58 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f82858c528808d2a9e77bb56f16e897cfb5bbe73 
  src/main/python/apache/aurora/executor/common/announcer.py 
34e36e0a59093468a8934f58bacb68512949347c 
  src/test/python/apache/aurora/executor/common/test_announcer.py 
f4032c7302f4733ab5670322b1905102c200f1c9 

Diff: https://reviews.apache.org/r/44532/diff/


Testing
---

Tested on vagrant with a wrapper for the executor


Thanks,

Kunal Thakar

Re: Review Request 44532: Allow overriding hostname before announcing

2016-03-19 Thread Stephan Erb 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44532/#review124244
---




docs/configuration-reference.md (line 472)


That sounds a little bit like "ip" is a valid value. Using `` 
might make this more clear.


- Stephan Erb


On March 18, 2016, 7:18 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44532/
> ---
> 
> (Updated March 18, 2016, 7:18 p.m.)
> 
> 
> Review request for Aurora, John Sirois, Stephan Erb, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Allow overriding hostname before announcing 
> (https://issues.apache.org/jira/browse/AURORA-1611)
> 
> 
> Diffs
> -
> 
>   RELEASE-NOTES.md 42bb340c492f25286c8c785885132abcbe455296 
>   docs/configuration-reference.md e043c4886c6d0fd96646f4687865d1c715cd1c58 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> f82858c528808d2a9e77bb56f16e897cfb5bbe73 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 34e36e0a59093468a8934f58bacb68512949347c 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> f4032c7302f4733ab5670322b1905102c200f1c9 
> 
> Diff: https://reviews.apache.org/r/44532/diff/
> 
> 
> Testing
> ---
> 
> Tested on vagrant with a wrapper for the executor
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 44532: Allow overriding hostname before announcing

2016-03-19 Thread Aurora ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44532/#review124076
---


Ship it!




Master (6537581) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot 
retry"

- Aurora ReviewBot


On March 17, 2016, 6:04 p.m., Kunal Thakar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44532/
> ---
> 
> (Updated March 17, 2016, 6:04 p.m.)
> 
> 
> Review request for Aurora, John Sirois, Stephan Erb, and Bill Farner.
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Allow overriding hostname before announcing 
> (https://issues.apache.org/jira/browse/AURORA-1611)
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
> f82858c528808d2a9e77bb56f16e897cfb5bbe73 
>   src/main/python/apache/aurora/executor/common/announcer.py 
> 34e36e0a59093468a8934f58bacb68512949347c 
>   src/test/python/apache/aurora/executor/common/test_announcer.py 
> f4032c7302f4733ab5670322b1905102c200f1c9 
> 
> Diff: https://reviews.apache.org/r/44532/diff/
> 
> 
> Testing
> ---
> 
> Tested on vagrant with a wrapper for the executor
> 
> 
> Thanks,
> 
> Kunal Thakar
> 
>

Re: Review Request 44871: Speed up Vagrant provisioning by using a custom base box.

2016-03-19 Thread Bill Farner 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44871/
---

(Updated March 16, 2016, 10:44 p.m.)


Review request for Aurora, John Sirois and Zameer Manji.


Changes
---

Speculating that a broken docker install in the base image was causing e2e test 
failures.  Updated docker in the base box and end-to-end tests are now green.


Repository: aurora


Description (updated)
---

Relevant docs on packer can be found here: https://www.packer.io/docs/

To support this, i have created the [`apache-aurora` 
org](https://atlas.hashicorp.com/apache-aurora).  If we move forward with this 
patch, i will be looking to add other committers to the org and improve the bus 
factor.


Diffs (updated)
-

  .gitignore 1af09a251b3f76c13813033d32aa7efba9aef304 
  Vagrantfile 2d6c2ae598e80035840f7e517e161be266b581dd 
  build-support/packer/README.md PRE-CREATION 
  build-support/packer/aurora.json PRE-CREATION 
  build-support/packer/build.sh PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-master/ip PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-master/roles PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/containerizers PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/hostname PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/ip PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/modules PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/resource_estimator PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/resources PRE-CREATION 
  examples/vagrant/mesos_config/etc_mesos-slave/work_dir PRE-CREATION 
  examples/vagrant/provision-dev-cluster.sh 
df33a30c1dcc0ea832899c849be05fe9deb47433 
  examples/vagrant/upstart/aurora-scheduler-kerberos.conf 
eec5ee3a2f041ae25fa664b80004e27ed8b5e2b0 
  examples/vagrant/upstart/aurora-scheduler.conf 
7e10b2c7690087bb150a9033cbd576ab82b1b185 
  examples/vagrant/upstart/mesos-master.conf 
a2ed953d871a7b28444a188ee0fc152853c5b088 
  examples/vagrant/upstart/mesos-slave.conf 
d97fa1fcf99580ef3f0997e9c78cb9775b91a818 

Diff: https://reviews.apache.org/r/44871/diff/


Testing
---

I can run `vagrant up`, end-to-end tests pass.

Reviewers - you can do this as well by applying the patch.  You will observe a 
long fetch on the first run (much like you would with the current repo if you 
wiped your local vagrant boxes).  However, subsequent `vagrant destroy`, 
`vagrant up` cycles should be relatively fast, about 1m30s for me.


Thanks,

Bill Farner