subject:"Re\: Review Request 28920\: Add support for docker containers to aurora"


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69439
---


@ReviewBot retry

- Steve Niemitz


On Jan. 23, 2015, 5:23 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 23, 2015, 5:23 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh f692f025a7e5f2f0dddb7f6c81ea12fcb8272020 
   examples/vagrant/provision-dev-cluster.sh 
 40c165925c2110fb727c66ff5a34cf5ab8415343 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 8428941c10641857a952f34df4e46a8fac5476a8 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5cc85f1f87f3b8355c89e8ecac19de1122a079e6 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 f29faf1d51baa4af66ad8c6579ffa354409e9536 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 41200bd5d74ee3239279567d79a5f48fb6af2e44 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 9fd188fb8f004d0a7664420bfda56568cebedb6f 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 c7de6e111300b009e1f9f430624a56100328184e 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 6c8ae1cf7fb578237708a954bd42162d66b39f4d 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-23 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69447
---

Ship it!


Master (3fa004b) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 23, 2015, 5:23 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 23, 2015, 5:23 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh f692f025a7e5f2f0dddb7f6c81ea12fcb8272020 
   examples/vagrant/provision-dev-cluster.sh 
 40c165925c2110fb727c66ff5a34cf5ab8415343 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 8428941c10641857a952f34df4e46a8fac5476a8 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5cc85f1f87f3b8355c89e8ecac19de1122a079e6 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 f29faf1d51baa4af66ad8c6579ffa354409e9536 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 41200bd5d74ee3239279567d79a5f48fb6af2e44 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 9fd188fb8f004d0a7664420bfda56568cebedb6f 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 c7de6e111300b009e1f9f430624a56100328184e 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 6c8ae1cf7fb578237708a954bd42162d66b39f4d 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-23 Thread Brian Wickman


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69480
---


this is looking great.  just a few small nits and one higher level question re: 
where the schema should live.


src/main/python/apache/aurora/executor/common/sandbox.py
https://reviews.apache.org/r/28920/#comment114179

minor nit: snake_case, i.e. _create_symlinks



src/main/python/apache/aurora/executor/common/sandbox.py
https://reviews.apache.org/r/28920/#comment114182

os.chdir is a surprising side-effect for sandbox.create.

could you instead put this in ThermosTaskRunner.start?  i.e. where we do 
self._popen = subprocess.Popen(cmdline_args), Popen takes a cwd= keyword 
argument, and it seems like the presence of $MESOS_DIRECTORY should be enough 
signal that we should be setting it, and we don't need to chdir here.



src/main/python/apache/aurora/executor/thermos_task_runner.py
https://reviews.apache.org/r/28920/#comment114178

minor nit: os.path.join(X, 'sandbox') instead of + '/sandbox'



src/main/python/apache/aurora/executor/thermos_task_runner.py
https://reviews.apache.org/r/28920/#comment114181

b/c of AURORA-749, I think this can now just be:

None if assigned_task.task.container.docker else assigned_task.task.job.role



src/main/python/apache/thermos/config/schema_base.py
https://reviews.apache.org/r/28920/#comment114180

sort of higher-level question -- since this information is not used by 
thermos at all, does it make sense in  
src/main/python/apache/aurora/config/schema/base.py instead?  usually Aurora 
concerns are in Job() and Thermos concerns are in Task().


- Brian Wickman


On Jan. 23, 2015, 5:23 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 23, 2015, 5:23 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh f692f025a7e5f2f0dddb7f6c81ea12fcb8272020 
   examples/vagrant/provision-dev-cluster.sh 
 40c165925c2110fb727c66ff5a34cf5ab8415343 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 8428941c10641857a952f34df4e46a8fac5476a8 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5cc85f1f87f3b8355c89e8ecac19de1122a079e6 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 f29faf1d51baa4af66ad8c6579ffa354409e9536 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-23 Thread Brian Wickman


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69505
---

Ship it!


sweet!  ship it modulo minor nit below


src/main/python/apache/aurora/executor/thermos_task_runner.py
https://reviews.apache.org/r/28920/#comment114201

os.environ.get('MESOS_DIRECTORY')

(i worry that cwd='.' will actually break something, whereas cwd=None will 
be a no-op.)


- Brian Wickman


On Jan. 24, 2015, midnight, Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 24, 2015, midnight)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh f692f025a7e5f2f0dddb7f6c81ea12fcb8272020 
   examples/vagrant/provision-dev-cluster.sh 
 40c165925c2110fb727c66ff5a34cf5ab8415343 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 8428941c10641857a952f34df4e46a8fac5476a8 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5cc85f1f87f3b8355c89e8ecac19de1122a079e6 
   src/main/python/apache/aurora/config/schema/base.py 
 d7897794c736778983d506c337a1392f3cc0cc20 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 f29faf1d51baa4af66ad8c6579ffa354409e9536 
   src/main/python/apache/thermos/core/runner.py 
 41200bd5d74ee3239279567d79a5f48fb6af2e44 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 9fd188fb8f004d0a7664420bfda56568cebedb6f 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 c7de6e111300b009e1f9f430624a56100328184e 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 6c8ae1cf7fb578237708a954bd42162d66b39f4d 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-23 Thread Kevin Sweeney


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69512
---

Ship it!


Ship It!

- Kevin Sweeney


On Jan. 23, 2015, 4 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 23, 2015, 4 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh f692f025a7e5f2f0dddb7f6c81ea12fcb8272020 
   examples/vagrant/provision-dev-cluster.sh 
 40c165925c2110fb727c66ff5a34cf5ab8415343 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 8428941c10641857a952f34df4e46a8fac5476a8 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5cc85f1f87f3b8355c89e8ecac19de1122a079e6 
   src/main/python/apache/aurora/config/schema/base.py 
 d7897794c736778983d506c337a1392f3cc0cc20 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 f29faf1d51baa4af66ad8c6579ffa354409e9536 
   src/main/python/apache/thermos/core/runner.py 
 41200bd5d74ee3239279567d79a5f48fb6af2e44 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 9fd188fb8f004d0a7664420bfda56568cebedb6f 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 c7de6e111300b009e1f9f430624a56100328184e 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 6c8ae1cf7fb578237708a954bd42162d66b39f4d 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  docs/deploying-aurora-scheduler.md, line 163
  https://reviews.apache.org/r/28920/diff/18/?file=823201#file823201line163
 
  Philosophical question: if there's already a hard requirement that the 
  container have Python 2.7 why not require that the executor be baked in as 
  well? Maybe it's worth calling out as a TODO, but you don't have to answer 
  it now.
 
 Steve Niemitz wrote:
 I think baking the executor into docker images is a recipe for disaster.  
 Any time you upgraded aurora you'd need to then go update all containers with 
 the new executor.  Also, I don't like the idea of having to build specific 
 aurora-isms into docker containers (I don't even really like requiring 
 python, but that's unavoidable).
 
 Bill Farner wrote:
 I'm with Steve here.  Forcing this seems overly restrictive.  However, i 
 would like to support the mode of operation you describe, Kevin.
 
 Kevin Sweeney wrote:
 Just seems weird to put partial requirements on the execution environment 
 of the container we can run, which somewhat defeats the purpose of containers.
 
 Filed https://issues.apache.org/jira/browse/AURORA-1051 to explore 
 removing this restriction.

I'd love to see that as well.


 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, 
  lines 147-158
  https://reviews.apache.org/r/28920/diff/18/?file=823211#file823211line147
 
  I don't see a reason the executor can't do this itself, by reading the 
  Container field in AssignedTask and environment variables. I'd prefer not 
  to introduce a new channel to send configuration from the scheduler to the 
  executor with this review.
  
  @wickman might be better able to answer whether this is feasible.
 
 Steve Niemitz wrote:
 Some of the things above can't be (easily) done here in the executor 
 because it's already to late by the time code in the executor begins running. 
  For example, logging is already initialized before any real app code runs.  
 This has gone back and forth a few times now, and we landed on keeping 
 anything docker specific out of the executor (which I like better anyways).  
 There's been a good amount of conversation about this in reviews above.

See below on this one.


 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  src/main/python/apache/thermos/core/runner.py, lines 625-632
  https://reviews.apache.org/r/28920/diff/18/?file=823218#file823218line625
 
  Naive question: since we have this block here can we drop the preamble 
  from the scheduler?
 
 Steve Niemitz wrote:
 We need both, they work together to get the logs/sandbox into the right 
 place, and allow the observer to pick it up.

Also see below, this moved around a little bit so it's not in thermos anymore.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68845
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 23, 2015, 4:25 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Changes
---

This includes a rebase from the latest master.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh f692f025a7e5f2f0dddb7f6c81ea12fcb8272020 
  examples/vagrant/provision-dev-cluster.sh 
40c165925c2110fb727c66ff5a34cf5ab8415343 
  examples/vagrant/upstart/aurora-scheduler.conf 
788ec254270bca074dae91829c7f4fccdc8f8bb0 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
8428941c10641857a952f34df4e46a8fac5476a8 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5cc85f1f87f3b8355c89e8ecac19de1122a079e6 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f7d8977e42aa56188799400bf8e12a6886fb4a8f 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
f29faf1d51baa4af66ad8c6579ffa354409e9536 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
41200bd5d74ee3239279567d79a5f48fb6af2e44 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
9fd188fb8f004d0a7664420bfda56568cebedb6f 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  
src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
 dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
c7de6e111300b009e1f9f430624a56100328184e 
  src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
7eafe074b686d55ad96018006cf4acfa823513c3 
  
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
 ad9126c32893080e128d086ea3bfd7ad23d27b89 
  src/test/python/apache/aurora/client/cli/test_status.py 
e531fa06e508d9792af51c62e67120c21baa7a81 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
6c8ae1cf7fb578237708a954bd42162d66b39f4d 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 10:42 p.m., Brian Wickman wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, 
  lines 153-158
  https://reviews.apache.org/r/28920/diff/18/?file=823211#file823211line153
 
  can't the stuff in DOCKER_COMMAND_PREFIX be accomplished with a 
  DockerSandboxProvider in the thermos executor?
  
  similarly, DOCKER_COMMAND_SUFFIX should be unnecessary since this 
  information is available in the TaskConfig.
  
  if both the above are true, then we don't need any of the changes to 
  CommandUtil.
 
 Steve Niemitz wrote:
 No unfortunatly, once the executor is started it's too late to change 
 some things, they need to be setup before the executor launches (see my 
 previous comment w/ Kevin). The suffix could be moved into the executor, but 
 again I'd like to keep docker specific logic out of there.
 
 Brian Wickman wrote:
 it's definitely not too late.  and i disagree that we should keep docker 
 specific logic out of the executor.  that's *exactly* where it should be.
 
 afaict, the only thing affected is LogOptions.set_log_dir('.') in 
 thermos_executor_main.  this can easily be changed to 
 LogOptions.set_log_dir(os.getenv('MESOS_DIRECTORY') or '.')
 
 *ideally* in my opinion, the logical conclusion is creating a 
 thermos_docker_executor_main that wires things up in accordance to the docker 
 philosophy, then have a completely separate docker task factory within the 
 scheduler to construct these.  but that might be a little more unsavory in 
 the short term than just having the thermos_executor that understands both 
 containerized and non-containerized modes.  there is precedence for this 
 already (exactly why we have sandbox providers and task providers in the 
 first place -- to switch implementations entirely based upon incoming 
 TaskConfigs.)
 
 Steve Niemitz wrote:
 I can investiate moving it into the executor, but building an entirely 
 seperate executor for docker is more than I'm willing to take on at this 
 point.
 
 Steve Niemitz wrote:
 I need to think about this more, using your suggestion of 
 LogOptions.set_log_dir(os.getenv('MESOS_DIRECTORY') or '.') fails because the 
 directory doesn't yet exist (since the symlink hasn't yet been created).  Its 
 possible we could defer setting up logging until the executor gets a task and 
 set everything up in aurora_executor, but I dont know what else that might 
 affect.

Ok I have something working here, MESOS_DIRECTORY into the logger doesn't work 
since it doesn't exist yet (and I don't want the logger to create it), but 
MESOS_SANDBOX does.  That plus a little extra in the DefaultSandboxProvider and 
removing references to . in thermos_executor_main seems to have done the 
trick.  Diff coming soon.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69288
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-22 Thread Bill Farner



 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  docs/deploying-aurora-scheduler.md, line 163
  https://reviews.apache.org/r/28920/diff/18/?file=823201#file823201line163
 
  Philosophical question: if there's already a hard requirement that the 
  container have Python 2.7 why not require that the executor be baked in as 
  well? Maybe it's worth calling out as a TODO, but you don't have to answer 
  it now.
 
 Steve Niemitz wrote:
 I think baking the executor into docker images is a recipe for disaster.  
 Any time you upgraded aurora you'd need to then go update all containers with 
 the new executor.  Also, I don't like the idea of having to build specific 
 aurora-isms into docker containers (I don't even really like requiring 
 python, but that's unavoidable).

I'm with Steve here.  Forcing this seems overly restrictive.  However, i would 
like to support the mode of operation you describe, Kevin.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68845
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69282
---



src/main/python/apache/thermos/core/runner.py
https://reviews.apache.org/r/28920/#comment113921

this is an abstraction leak.  grep the thermos codebase for 'aurora' and 
'mesos'.  thermos should remain aurora/mesos agnostic.

thermos_runner.py includes a --sandbox flag and we should pass 
$MESOS_DIRECTORY to that, and not override here.  the log_dir defaults to 
os.path.join(sandbox, '.logs'), so the log_dir part here is unnecessary.


- Brian Wickman


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  docs/deploying-aurora-scheduler.md, line 163
  https://reviews.apache.org/r/28920/diff/18/?file=823201#file823201line163
 
  Philosophical question: if there's already a hard requirement that the 
  container have Python 2.7 why not require that the executor be baked in as 
  well? Maybe it's worth calling out as a TODO, but you don't have to answer 
  it now.

I think baking the executor into docker images is a recipe for disaster.  Any 
time you upgraded aurora you'd need to then go update all containers with the 
new executor.  Also, I don't like the idea of having to build specific 
aurora-isms into docker containers (I don't even really like requiring python, 
but that's unavoidable).


 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  examples/vagrant/provision-dev-cluster.sh, line 17
  https://reviews.apache.org/r/28920/diff/18/?file=823204#file823204line17
 
  nit: sh -c indirection is unnecessary here.

Eh, this is just copied from the [docker install 
docs](https://docs.docker.com/installation/ubuntulinux/#ubuntu-precise-1204-lts-64-bit).


 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  src/main/python/apache/thermos/core/runner.py, lines 625-632
  https://reviews.apache.org/r/28920/diff/18/?file=823218#file823218line625
 
  Naive question: since we have this block here can we drop the preamble 
  from the scheduler?

We need both, they work together to get the logs/sandbox into the right place, 
and allow the observer to pick it up.


 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, 
  lines 147-158
  https://reviews.apache.org/r/28920/diff/18/?file=823211#file823211line147
 
  I don't see a reason the executor can't do this itself, by reading the 
  Container field in AssignedTask and environment variables. I'd prefer not 
  to introduce a new channel to send configuration from the scheduler to the 
  executor with this review.
  
  @wickman might be better able to answer whether this is feasible.

Some of the things above can't be (easily) done here in the executor because 
it's already to late by the time code in the executor begins running.  For 
example, logging is already initialized before any real app code runs.  This 
has gone back and forth a few times now, and we landed on keeping anything 
docker specific out of the executor (which I like better anyways).  There's 
been a good amount of conversation about this in reviews above.


 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  src/main/python/apache/thermos/config/schema_base.py, lines 65-70
  https://reviews.apache.org/r/28920/diff/18/?file=823217#file823217line65
 
  Can we match the union-like behavior the IDL defines here,
  
  something like:
  
  ```py
  class Container(Struct):
docker = Docker

  class Docker(Struct):
image = Required(String)
  
  ```
  
  This will be more easily extensible to more container types without 
  requiring backwards-incompatible changes.

I like this idea.  I'll make the changes.


 On Jan. 22, 2015, 2:35 a.m., Kevin Sweeney wrote:
  docs/deploying-aurora-scheduler.md, line 158
  https://reviews.apache.org/r/28920/diff/18/?file=823201#file823201line158
 
  Latest version doesn't require a wrapper script, update docs to reflect?

The previous sentence here is If using a wrapper script.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68845
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 10:22 p.m., Brian Wickman wrote:
  src/main/python/apache/thermos/core/runner.py, lines 627-632
  https://reviews.apache.org/r/28920/diff/18/?file=823218#file823218line627
 
  this is an abstraction leak.  grep the thermos codebase for 'aurora' 
  and 'mesos'.  thermos should remain aurora/mesos agnostic.
  
  thermos_runner.py includes a --sandbox flag and we should pass 
  $MESOS_DIRECTORY to that, and not override here.  the log_dir defaults to 
  os.path.join(sandbox, '.logs'), so the log_dir part here is unnecessary.

Heh, I'm pretty sure it was like that ~20 reviews ago.  To be clear, your 
proposing passing the flag into the runner from thermos_task_runner.py?


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69282
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 10:22 p.m., Brian Wickman wrote:
  src/main/python/apache/thermos/core/runner.py, lines 627-632
  https://reviews.apache.org/r/28920/diff/18/?file=823218#file823218line627
 
  this is an abstraction leak.  grep the thermos codebase for 'aurora' 
  and 'mesos'.  thermos should remain aurora/mesos agnostic.
  
  thermos_runner.py includes a --sandbox flag and we should pass 
  $MESOS_DIRECTORY to that, and not override here.  the log_dir defaults to 
  os.path.join(sandbox, '.logs'), so the log_dir part here is unnecessary.
 
 Steve Niemitz wrote:
 Heh, I'm pretty sure it was like that ~20 reviews ago.  To be clear, your 
 proposing passing the flag into the runner from thermos_task_runner.py?

yep.


- Brian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69282
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-22 Thread Jay Buffington


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69333
---



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment114008

Can you set force_pull_image to true here.  I can't imagine why you would 
ever want that to be false. 
https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L1028


- Jay Buffington


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 23, 2015, 2:43 a.m., Jay Buffington wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, line 
  294
  https://reviews.apache.org/r/28920/diff/18/?file=823211#file823211line294
 
  Can you set force_pull_image to true here.  I can't imagine why you 
  would ever want that to be false. 
  https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L1028

I have plans for that along with privilaged mode, but both require mesos 0.21.0 
:(  Once this patch is in we can start the discussion of bumping the 
requirement up on mesos.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69333
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 10:42 p.m., Brian Wickman wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, 
  lines 153-158
  https://reviews.apache.org/r/28920/diff/18/?file=823211#file823211line153
 
  can't the stuff in DOCKER_COMMAND_PREFIX be accomplished with a 
  DockerSandboxProvider in the thermos executor?
  
  similarly, DOCKER_COMMAND_SUFFIX should be unnecessary since this 
  information is available in the TaskConfig.
  
  if both the above are true, then we don't need any of the changes to 
  CommandUtil.

No unfortunatly, once the executor is started it's too late to change some 
things, they need to be setup before the executor launches (see my previous 
comment w/ Kevin). The suffix could be moved into the executor, but again I'd 
like to keep docker specific logic out of there.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69288
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-22 Thread Kevin Sweeney



 On Jan. 21, 2015, 6:35 p.m., Kevin Sweeney wrote:
  examples/vagrant/provision-dev-cluster.sh, line 17
  https://reviews.apache.org/r/28920/diff/18/?file=823204#file823204line17
 
  nit: sh -c indirection is unnecessary here.
 
 Steve Niemitz wrote:
 Eh, this is just copied from the [docker install 
 docs](https://docs.docker.com/installation/ubuntulinux/#ubuntu-precise-1204-lts-64-bit).

The `sh -c` indirection is necessary there because of the use of `sudo`. In a 
script that's already running as root it's unnecesary.


 On Jan. 21, 2015, 6:35 p.m., Kevin Sweeney wrote:
  docs/deploying-aurora-scheduler.md, line 163
  https://reviews.apache.org/r/28920/diff/18/?file=823201#file823201line163
 
  Philosophical question: if there's already a hard requirement that the 
  container have Python 2.7 why not require that the executor be baked in as 
  well? Maybe it's worth calling out as a TODO, but you don't have to answer 
  it now.
 
 Steve Niemitz wrote:
 I think baking the executor into docker images is a recipe for disaster.  
 Any time you upgraded aurora you'd need to then go update all containers with 
 the new executor.  Also, I don't like the idea of having to build specific 
 aurora-isms into docker containers (I don't even really like requiring 
 python, but that's unavoidable).
 
 Bill Farner wrote:
 I'm with Steve here.  Forcing this seems overly restrictive.  However, i 
 would like to support the mode of operation you describe, Kevin.

Just seems weird to put partial requirements on the execution environment of 
the container we can run, which somewhat defeats the purpose of containers.

Filed https://issues.apache.org/jira/browse/AURORA-1051 to explore removing 
this restriction.


 On Jan. 21, 2015, 6:35 p.m., Kevin Sweeney wrote:
  docs/deploying-aurora-scheduler.md, line 158
  https://reviews.apache.org/r/28920/diff/18/?file=823201#file823201line158
 
  Latest version doesn't require a wrapper script, update docs to reflect?
 
 Steve Niemitz wrote:
 The previous sentence here is If using a wrapper script.

Gotcha, my bad.


- Kevin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68845
---


On Jan. 15, 2015, 4:08 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 15, 2015, 4:08 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69288
---



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment113931

can't the stuff in DOCKER_COMMAND_PREFIX be accomplished with a 
DockerSandboxProvider in the thermos executor?

similarly, DOCKER_COMMAND_SUFFIX should be unnecessary since this 
information is available in the TaskConfig.

if both the above are true, then we don't need any of the changes to 
CommandUtil.


- Brian Wickman


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 10:42 p.m., Brian Wickman wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, 
  lines 153-158
  https://reviews.apache.org/r/28920/diff/18/?file=823211#file823211line153
 
  can't the stuff in DOCKER_COMMAND_PREFIX be accomplished with a 
  DockerSandboxProvider in the thermos executor?
  
  similarly, DOCKER_COMMAND_SUFFIX should be unnecessary since this 
  information is available in the TaskConfig.
  
  if both the above are true, then we don't need any of the changes to 
  CommandUtil.
 
 Steve Niemitz wrote:
 No unfortunatly, once the executor is started it's too late to change 
 some things, they need to be setup before the executor launches (see my 
 previous comment w/ Kevin). The suffix could be moved into the executor, but 
 again I'd like to keep docker specific logic out of there.

it's definitely not too late.  and i disagree that we should keep docker 
specific logic out of the executor.  that's *exactly* where it should be.

afaict, the only thing affected is LogOptions.set_log_dir('.') in 
thermos_executor_main.  this can easily be changed to 
LogOptions.set_log_dir(os.getenv('MESOS_DIRECTORY') or '.')

*ideally* in my opinion, the logical conclusion is creating a 
thermos_docker_executor_main that wires things up in accordance to the docker 
philosophy, then have a completely separate docker task factory within the 
scheduler to construct these.  but that might be a little more unsavory in the 
short term than just having the thermos_executor that understands both 
containerized and non-containerized modes.  there is precedence for this 
already (exactly why we have sandbox providers and task providers in the first 
place -- to switch implementations entirely based upon incoming TaskConfigs.)


- Brian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69288
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 22, 2015, 10:42 p.m., Brian Wickman wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, 
  lines 153-158
  https://reviews.apache.org/r/28920/diff/18/?file=823211#file823211line153
 
  can't the stuff in DOCKER_COMMAND_PREFIX be accomplished with a 
  DockerSandboxProvider in the thermos executor?
  
  similarly, DOCKER_COMMAND_SUFFIX should be unnecessary since this 
  information is available in the TaskConfig.
  
  if both the above are true, then we don't need any of the changes to 
  CommandUtil.
 
 Steve Niemitz wrote:
 No unfortunatly, once the executor is started it's too late to change 
 some things, they need to be setup before the executor launches (see my 
 previous comment w/ Kevin). The suffix could be moved into the executor, but 
 again I'd like to keep docker specific logic out of there.
 
 Brian Wickman wrote:
 it's definitely not too late.  and i disagree that we should keep docker 
 specific logic out of the executor.  that's *exactly* where it should be.
 
 afaict, the only thing affected is LogOptions.set_log_dir('.') in 
 thermos_executor_main.  this can easily be changed to 
 LogOptions.set_log_dir(os.getenv('MESOS_DIRECTORY') or '.')
 
 *ideally* in my opinion, the logical conclusion is creating a 
 thermos_docker_executor_main that wires things up in accordance to the docker 
 philosophy, then have a completely separate docker task factory within the 
 scheduler to construct these.  but that might be a little more unsavory in 
 the short term than just having the thermos_executor that understands both 
 containerized and non-containerized modes.  there is precedence for this 
 already (exactly why we have sandbox providers and task providers in the 
 first place -- to switch implementations entirely based upon incoming 
 TaskConfigs.)
 
 Steve Niemitz wrote:
 I can investiate moving it into the executor, but building an entirely 
 seperate executor for docker is more than I'm willing to take on at this 
 point.

I need to think about this more, using your suggestion of 
LogOptions.set_log_dir(os.getenv('MESOS_DIRECTORY') or '.') fails because the 
directory doesn't yet exist (since the symlink hasn't yet been created).  Its 
possible we could defer setting up logging until the executor gets a task and 
set everything up in aurora_executor, but I dont know what else that might 
affect.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review69288
---


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-21 Thread Kevin Sweeney


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68845
---


Sorry for the delay in reply. I've been thinking a lot about this patch and 
it's really looking good. My comments are focused mostly on 
backwards-compatibility as well as the desire to not favor Docker over other 
future containerizers Mesos may add.


docs/deploying-aurora-scheduler.md
https://reviews.apache.org/r/28920/#comment113341

Latest version doesn't require a wrapper script, update docs to reflect?



docs/deploying-aurora-scheduler.md
https://reviews.apache.org/r/28920/#comment113339

Philosophical question: if there's already a hard requirement that the 
container have Python 2.7 why not require that the executor be baked in as 
well? Maybe it's worth calling out as a TODO, but you don't have to answer it 
now.



examples/vagrant/provision-dev-cluster.sh
https://reviews.apache.org/r/28920/#comment113340

nit: sh -c indirection is unnecessary here.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment113726

I don't see a reason the executor can't do this itself, by reading the 
Container field in AssignedTask and environment variables. I'd prefer not to 
introduce a new channel to send configuration from the scheduler to the 
executor with this review.

@wickman might be better able to answer whether this is feasible.



src/main/python/apache/thermos/config/schema_base.py
https://reviews.apache.org/r/28920/#comment113732

Can we match the union-like behavior the IDL defines here,

something like:

```py
class Container(Struct):
  docker = Docker
  
class Docker(Struct):
  image = Required(String)

```

This will be more easily extensible to more container types without 
requiring backwards-incompatible changes.



src/main/python/apache/thermos/core/runner.py
https://reviews.apache.org/r/28920/#comment113733

Naive question: since we have this block here can we drop the preamble from 
the scheduler?


- Kevin Sweeney


On Jan. 15, 2015, 4:08 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 15, 2015, 4:08 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68267
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 110, in resolve
new_requirements.update(requires(highest_package, requirement))
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 84, in requires
raise Untranslateable('Package %s is not translateable.' % package)
Untranslateable: Package 
WheelPackage(u'https://pypi.python.org/packages/py2.py3/p/pytest-timeout/pytest_timeout-0.4-py2.py3-none-any.whl#md5=84a0fa6b871f0c93b77206ed8840144d')
 is not translateable.


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68260
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

:licenseTest UP-TO-DATE
:license UP-TO-DATE
:pmdMain
:test
:jacocoTestReport
Coverage report generated: 
file:///x1/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/jacoco/test/html/index.html
:analyzeReport
Instruction coverage of 0.893543617545589 exceeds minimum coverage of 0.89.
Branch coverage of 0.8391933815925543 exceeds minimum coverage of 0.835.
:check
:build
:api:assemble
:api:compileTestJava UP-TO-DATE
:api:processTestResources UP-TO-DATE
:api:testClasses UP-TO-DATE
:api:test UP-TO-DATE
:api:check UP-TO-DATE
:api:build
:buildSrc:compileJava UP-TO-DATE
:buildSrc:processResources UP-TO-DATE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar
:buildSrc:assemble
:buildSrc:compileTestJava UP-TO-DATE
:buildSrc:processTestResources UP-TO-DATE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test UP-TO-DATE
:buildSrc:check UP-TO-DATE
:buildSrc:build

BUILD SUCCESSFUL

Total time: 3 mins 56.511 secs
+ export PIP_DEFAULT_TIMEOUT=60
+ PIP_DEFAULT_TIMEOUT=60
+ mkdir -p third_party
+ pip install -d third_party -r /dev/fd/63
++ grep -v mesos.native 3rdparty/python/requirements.txt
./build-support/jenkins/build.sh: line 28: pip: command not found
grep: write error: Broken pipe


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68247
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh


Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.string-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.threading
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.threading-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.util
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.util-0.3.1-py2.7-nspkg.pth
  Found existing installation: setuptools 3.6
Uninstalling setuptools:
  Successfully uninstalled setuptools
Successfully installed Markdown Pygments ansicolors cov-core coverage lockfile 
pantsbuild.pants pex psutil py pystache pytest pytest-cov python-daemon 
requests twitter.common.collections twitter.common.config 
twitter.common.confluence twitter.common.contextutil twitter.common.decorators 
twitter.common.dirutil twitter.common.lang twitter.common.log 
twitter.common.options twitter.common.process twitter.common.string 
twitter.common.threading twitter.common.util setuptools
Cleaning up...
Unable to detect suitable interpreter.

Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-15 Thread Steve Niemitz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 15, 2015, 8:24 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Changes
---

Added -allowed_container_types to scheduler and decoupled Aurora config schema 
from Mesos for container configuration.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
  examples/vagrant/provision-dev-cluster.sh 
7af4b52a6876268a97630279221bb98d9b04efad 
  examples/vagrant/upstart/aurora-scheduler.conf 
788ec254270bca074dae91829c7f4fccdc8f8bb0 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/configuration/ContainerType.java 
PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f7d8977e42aa56188799400bf8e12a6886fb4a8f 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  
src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
 dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
7eafe074b686d55ad96018006cf4acfa823513c3 
  
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
 ad9126c32893080e128d086ea3bfd7ad23d27b89 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68301
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68350
---


Master (1acb428) is red with this patch.
  ./build-support/jenkins/build.sh

Note: Recompile with -Xlint:unchecked for details.

:api:checkPython
:api:generateThriftEntitiesJava
:api:classesThriftEntities
:api:compileJava UP-TO-DATE
:api:generateThriftResources
:api:processResources UP-TO-DATE
:api:classes
:api:jar
:compileJavaNote: Writing 
file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/com/twitter/common/args/apt/cmdline.arg.info.txt.2

:processResources
:classes
:jar
:assemble
:compileJmhJavawarning: Supported source version 'RELEASE_6' from annotation 
processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
'1.7'
1 warning

:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh
:jsHint
'Mode' is defined but never used. 
(/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/resources/scheduler/assets/js/services.js:18:51)
 JobUpdateStatus: false, Mode: false */

:jsHint FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jsHint'.
 Process 'command '/home/jenkins/tools/java/jdk1.7.0_25-32/bin/java'' finished 
 with non-zero exit value 2

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

BUILD FAILED

Total time: 1 mins 31.514 secs


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 15, 2015, 8:24 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 15, 2015, 8:24 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/configuration/ContainerType.java 
 PRE-CREATION 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-15 Thread Steve Niemitz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 16, 2015, 12:08 a.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
  examples/vagrant/provision-dev-cluster.sh 
7af4b52a6876268a97630279221bb98d9b04efad 
  examples/vagrant/upstart/aurora-scheduler.conf 
788ec254270bca074dae91829c7f4fccdc8f8bb0 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f7d8977e42aa56188799400bf8e12a6886fb4a8f 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  
src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
 dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
7eafe074b686d55ad96018006cf4acfa823513c3 
  
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
 ad9126c32893080e128d086ea3bfd7ad23d27b89 
  src/test/python/apache/aurora/client/cli/test_status.py 
e531fa06e508d9792af51c62e67120c21baa7a81 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68237
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68232
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68223
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_builder.py,
 line 43, in build
debug=debug).run()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 101, in resolve
raise Unsatisfiable('Cannot satisfy requirements: %s' % 
requirement_set[requirement.key])
Unsatisfiable: Cannot satisfy requirements: 
[Requirement.parse('coverage=3.7.1,4.0a1')]


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68239
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 110, in resolve
new_requirements.update(requires(highest_package, requirement))
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 84, in requires
raise Untranslateable('Package %s is not translateable.' % package)
Untranslateable: Package 
SourcePackage(u'https://pypi.python.org/packages/source/t/twitter.common.rpc/twitter.common.rpc-0.3.2.tar.gz#md5=acf6d28f63bbdfdf9c4ddb1423883da2')
 is not translateable.


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68241
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68245
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68377
---

Ship it!


Master (b75ed0f) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 16, 2015, 12:08 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 16, 2015, 12:08 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 
 7eafe074b686d55ad96018006cf4acfa823513c3 
   
 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
  ad9126c32893080e128d086ea3bfd7ad23d27b89 
   src/test/python/apache/aurora/client/cli/test_status.py 
 e531fa06e508d9792af51c62e67120c21baa7a81 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-14 Thread Bill Farner



 On Jan. 14, 2015, 7:55 p.m., Bill Farner wrote:
  All nits this round.  Last issue for me, as you point out, is 
  validation/restriction of `ContainerType` in `ConfigurationManager.
  
  Doing a pass on test coverage in the meantime.
 
 Bill Farner wrote:
 Test coverage LGTM when supplemented with 
 https://reviews.apache.org/r/29827/
 
 Steve Niemitz wrote:
 Is there any need to validate anymore now that the volume configuration 
 is gone?  That's what was going to be validated correct?

I would like to see an argument to specify which `ContainerType`s the scheduler 
should allow.  This way the scheduler will not fatefully accept jobs and launch 
tasks that are not supported by the slave's command line and environment.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68098
---


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-14 Thread Bill Farner



 On Jan. 14, 2015, 7:55 p.m., Bill Farner wrote:
  All nits this round.  Last issue for me, as you point out, is 
  validation/restriction of `ContainerType` in `ConfigurationManager.
  
  Doing a pass on test coverage in the meantime.
 
 Bill Farner wrote:
 Test coverage LGTM when supplemented with 
 https://reviews.apache.org/r/29827/
 
 Steve Niemitz wrote:
 Is there any need to validate anymore now that the volume configuration 
 is gone?  That's what was going to be validated correct?
 
 Bill Farner wrote:
 I would like to see an argument to specify which `ContainerType`s the 
 scheduler should allow.  This way the scheduler will not fatefully accept 
 jobs and launch tasks that are not supported by the slave's command line and 
 environment.
 
 Steve Niemitz wrote:
 Do you have any ideas on how to plumb that into the ConfigurationManager? 
  Since it's a static class.

I replied above with a suggestion.  Copying here to save you from excessive 
clicking in reviewboard:

 Do you have any thoughts on how to get the setting propegated into the 
 ConfigurationManager?

We have some precedent for defining them as static fields in classes like this, 
but have started to backpedal on that as it is hostile to unit testing.  
Perhaps you can create a ConfigurationSettings class, to be injected into 
SchedulerThriftInterface and passed to ConfigurationManager?  One thing to keep 
in mind - there are two primary entrypoints to ConfigurationManager - one for 
thrift, and the other for storage recovery.  The thrift entrypoint tends to do 
validation and storage recovery mostly does backfilling of fields, to make sure 
that we continue to run jobs we have previously accepted.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68098
---


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 14, 2015, 7:55 p.m., Bill Farner wrote:
  All nits this round.  Last issue for me, as you point out, is 
  validation/restriction of `ContainerType` in `ConfigurationManager.
  
  Doing a pass on test coverage in the meantime.
 
 Bill Farner wrote:
 Test coverage LGTM when supplemented with 
 https://reviews.apache.org/r/29827/

Is there any need to validate anymore now that the volume configuration is 
gone?  That's what was going to be validated correct?


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68098
---


On Jan. 14, 2015, 7:14 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 7:14 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68125
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh


Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.string-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.threading
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.threading-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.util
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.util-0.3.1-py2.7-nspkg.pth
  Found existing installation: setuptools 3.6
Uninstalling setuptools:
  Successfully uninstalled setuptools
Successfully installed Markdown Pygments ansicolors cov-core coverage lockfile 
pantsbuild.pants pex psutil py pystache pytest pytest-cov python-daemon 
requests twitter.common.collections twitter.common.config 
twitter.common.confluence twitter.common.contextutil twitter.common.decorators 
twitter.common.dirutil twitter.common.lang twitter.common.log 
twitter.common.options twitter.common.process twitter.common.string 
twitter.common.threading twitter.common.util setuptools
Cleaning up...
Unable to detect suitable interpreter.

Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 14, 2015, 8:20 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
  examples/vagrant/provision-dev-cluster.sh 
7af4b52a6876268a97630279221bb98d9b04efad 
  examples/vagrant/upstart/aurora-scheduler.conf 
788ec254270bca074dae91829c7f4fccdc8f8bb0 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f7d8977e42aa56188799400bf8e12a6886fb4a8f 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  
src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
 dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 14, 2015, 7:55 p.m., Bill Farner wrote:
  All nits this round.  Last issue for me, as you point out, is 
  validation/restriction of `ContainerType` in `ConfigurationManager.
  
  Doing a pass on test coverage in the meantime.
 
 Bill Farner wrote:
 Test coverage LGTM when supplemented with 
 https://reviews.apache.org/r/29827/
 
 Steve Niemitz wrote:
 Is there any need to validate anymore now that the volume configuration 
 is gone?  That's what was going to be validated correct?
 
 Bill Farner wrote:
 I would like to see an argument to specify which `ContainerType`s the 
 scheduler should allow.  This way the scheduler will not fatefully accept 
 jobs and launch tasks that are not supported by the slave's command line and 
 environment.

Do you have any ideas on how to plumb that into the ConfigurationManager?  
Since it's a static class.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68098
---


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68112
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 110, in resolve
new_requirements.update(requires(highest_package, requirement))
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 84, in requires
raise Untranslateable('Package %s is not translateable.' % package)
Untranslateable: Package 
WheelPackage(u'https://pypi.python.org/packages/2.7/r/requests/requests-2.3.0-py2.py3-none-any.whl#md5=f2d850fd48fc10a93aa03d69b87b96b4')
 is not translateable.


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 7:14 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 7:14 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68132
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68155
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 110, in resolve
new_requirements.update(requires(highest_package, requirement))
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 84, in requires
raise Untranslateable('Package %s is not translateable.' % package)
Untranslateable: Package 
SourcePackage(u'https://pypi.python.org/packages/source/t/twitter.common.dirutil/twitter.common.dirutil-0.3.2.tar.gz#md5=8f13bf805cd55dcdfe0d4dc8ea507982')
 is not translateable.


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68185
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

:licenseTest UP-TO-DATE
:license UP-TO-DATE
:pmdMain
:test
:jacocoTestReport
Coverage report generated: 
file:///x1/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/jacoco/test/html/index.html
:analyzeReport
Instruction coverage of 0.8935905559858246 exceeds minimum coverage of 0.89.
Branch coverage of 0.8402275077559462 exceeds minimum coverage of 0.835.
:check
:build
:api:assemble
:api:compileTestJava UP-TO-DATE
:api:processTestResources UP-TO-DATE
:api:testClasses UP-TO-DATE
:api:test UP-TO-DATE
:api:check UP-TO-DATE
:api:build
:buildSrc:compileJava UP-TO-DATE
:buildSrc:processResources UP-TO-DATE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar
:buildSrc:assemble
:buildSrc:compileTestJava UP-TO-DATE
:buildSrc:processTestResources UP-TO-DATE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test UP-TO-DATE
:buildSrc:check UP-TO-DATE
:buildSrc:build

BUILD SUCCESSFUL

Total time: 3 mins 0.215 secs
+ export PIP_DEFAULT_TIMEOUT=60
+ PIP_DEFAULT_TIMEOUT=60
+ mkdir -p third_party
+ pip install -d third_party -r /dev/fd/63
++ grep -v mesos.native 3rdparty/python/requirements.txt
./build-support/jenkins/build.sh: line 28: pip: command not found
grep: write error: Broken pipe


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68203
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

:licenseTest UP-TO-DATE
:license UP-TO-DATE
:pmdMain
:test
:jacocoTestReport
Coverage report generated: 
file:///x1/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/jacoco/test/html/index.html
:analyzeReport
Instruction coverage of 0.893543617545589 exceeds minimum coverage of 0.89.
Branch coverage of 0.8391933815925543 exceeds minimum coverage of 0.835.
:check
:build
:api:assemble
:api:compileTestJava UP-TO-DATE
:api:processTestResources UP-TO-DATE
:api:testClasses UP-TO-DATE
:api:test UP-TO-DATE
:api:check UP-TO-DATE
:api:build
:buildSrc:compileJava UP-TO-DATE
:buildSrc:processResources UP-TO-DATE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar
:buildSrc:assemble
:buildSrc:compileTestJava UP-TO-DATE
:buildSrc:processTestResources UP-TO-DATE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test UP-TO-DATE
:buildSrc:check UP-TO-DATE
:buildSrc:build

BUILD SUCCESSFUL

Total time: 2 mins 59.342 secs
+ export PIP_DEFAULT_TIMEOUT=60
+ PIP_DEFAULT_TIMEOUT=60
+ mkdir -p third_party
+ pip install -d third_party -r /dev/fd/63
++ grep -v mesos.native 3rdparty/python/requirements.txt
./build-support/jenkins/build.sh: line 28: pip: command not found
grep: write error: Broken pipe


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68196
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 110, in resolve
new_requirements.update(requires(highest_package, requirement))
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 84, in requires
raise Untranslateable('Package %s is not translateable.' % package)
Untranslateable: Package 
SourcePackage(u'https://pypi.python.org/packages/source/t/thrift/thrift-0.9.1.tar.gz#md5=8989a8a96b0e3a3380cfb89c44e172a6')
 is not translateable.


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68210
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68200
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68213
---

Ship it!


Master (a350982) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68219
---


Master (a350982) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_builder.py,
 line 43, in build
debug=debug).run()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 101, in resolve
raise Unsatisfiable('Cannot satisfy requirements: %s' % 
requirement_set[requirement.key])
Unsatisfiable: Cannot satisfy requirements: [Requirement.parse('py=1.4.22')]


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 14, 2015, 8:20 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 8:20 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 14, 2015, 2:28 a.m., Kevin Sweeney wrote:
  examples/jobs/docker/hello_docker.aurora, line 5
  https://reviews.apache.org/r/28920/diff/13/?file=818518#file818518line5
 
  since we're supposedly running in the python2.7 container could we show 
  our version? Bonus points, use a python3 container to prove that this is 
  different from the host version?

If it were a python 3 container it would also need python 2.7 in order to run 
the executor.  This limitation is called out in the docs in deploying aurora 
scheduler under Considerations for running jobs in docker containers however 
I'll add it as a note in the configuration reference as well.


 On Jan. 14, 2015, 2:28 a.m., Kevin Sweeney wrote:
  examples/vagrant/aurorabuild.sh, line 74
  https://reviews.apache.org/r/28920/diff/13/?file=818519#file818519line74
 
  is this wrapper necessary anymore?

Nope, we can get rid of it.


 On Jan. 14, 2015, 2:28 a.m., Kevin Sweeney wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, 
  lines 297-298
  https://reviews.apache.org/r/28920/diff/13/?file=818527#file818527line297
 
  How would you feel about removing these suffixes from the scheduler and 
  letting the user specify an executor-per-container-type? I can see a 
  deployment scenario where the executor is baked into the docker containers.

I'll add a TODO here, but for the intial release here I don't think it's 
required to support that use case.


 On Jan. 14, 2015, 2:28 a.m., Kevin Sweeney wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, line 
  328
  https://reviews.apache.org/r/28920/diff/13/?file=818527#file818527line328
 
  why the wrap/unwrap combination here?

Thrift - Protobuf conversion.  Is there a better way to do it?  The 
ContainerInfo comment above will be the same.


 On Jan. 14, 2015, 2:28 a.m., Kevin Sweeney wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, line 
  336
  https://reviews.apache.org/r/28920/diff/13/?file=818527#file818527line336
 
  Can the thermos observer detect container checkpoints any other way? 
  IMO that would be preferable to plumbing it in on the scheduler-side.

Not without big changes to the observer.  This can go away if 
https://issues.apache.org/jira/browse/AURORA-708 is implemented.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68013
---


On Jan. 13, 2015, 1:11 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 13, 2015, 1:11 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68062
---


Third time's the charm?  I feel like I'm trying to shift without using a 
clutch...

@ReviewBot retry

- Steve Niemitz


On Jan. 14, 2015, 4:50 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 14, 2015, 4:50 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-13 Thread Kevin Sweeney


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review68013
---


Partial review, looks good but some questions on the scheduler side of the code.


docs/configuration-reference.md
https://reviews.apache.org/r/28920/#comment112145

Qualify that this only applies to the docker containerizer?



examples/jobs/docker/hello_docker.aurora
https://reviews.apache.org/r/28920/#comment112150

since we're supposedly running in the python2.7 container could we show our 
version? Bonus points, use a python3 container to prove that this is different 
from the host version?



examples/vagrant/aurorabuild.sh
https://reviews.apache.org/r/28920/#comment112151

is this wrapper necessary anymore?



src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
https://reviews.apache.org/r/28920/#comment112152

nit: 2sp here.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112153

Use the Guava Splitter class here (but see Bill's comment above re: letting 
Args do this for us).



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112154

no need for this loop - protobuf provides

.addAllResources()



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112155

boolean - prefer value types to avoid nullability.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112156

boolean here



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112157

Why not derive this from the ContainerType enum from the TaskConfig?



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112158

How would you feel about removing these suffixes from the scheduler and 
letting the user specify an executor-per-container-type? I can see a deployment 
scenario where the executor is baked into the docker containers.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112160

why the wrap/unwrap combination here?



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112159

Can the thermos observer detect container checkpoints any other way? IMO 
that would be preferable to plumbing it in on the scheduler-side.


- Kevin Sweeney


On Jan. 12, 2015, 5:11 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 12, 2015, 5:11 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-13 Thread Bill Farner


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67985
---


Mostly style and configuration ergonomics remaining.  Once these are addressed 
my final pass before giving a ship will be assessing for complete test coverage.


docs/configuration-reference.md
https://reviews.apache.org/r/28920/#comment112074

Flag as experimental somehow, here and in other new doc portions.



examples/vagrant/upstart/aurora-scheduler.conf
https://reviews.apache.org/r/28920/#comment112075

remove extra newline



examples/vagrant/upstart/mesos-slave.conf
https://reviews.apache.org/r/28920/#comment112076

Is this afflicted by MESOS-2215?  I'm worried (read: certain) people will 
copy/paste the example here, and likely be set up for failure.  I don't have an 
answer, curious what your thoughts are.



src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
https://reviews.apache.org/r/28920/#comment112077

The Args system handles multi-valued args, comma-separated.  Change to 
`ArgListString` and this will Just Work.



src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
https://reviews.apache.org/r/28920/#comment112107

Should this instead be a list of allowed `ContainerType`s?  If we want this 
to support arbitrary container types, seems like a good litmus test for success 
is absense of the string 'docker' from code and comments.  Of course, there may 
be special cases that dictate otherwise.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112088

Please grep for 'wrapper' to make sure that code, comments, and docs refer 
to this as executor resources.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112084

This one is not optional.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112083

Feel free to omit A[n] optional from all of these, as the type 
self-documents.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112078

blank line between wrapped method signature and body.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112079

space-pad the colon:

for (String resource : splitResources) {



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment112089

shell=true is default, is there a reason you want to explicitly set it?  
I'm not against it, just want to be on the same page.



src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
https://reviews.apache.org/r/28920/#comment112115

I would love to see another check for whether the scheduler is configured 
to allow docker mounts.  In fact, i'm not sure if `allow_docker_mounts` (or its 
replacement, based on another comment) should be plumbed anywhere else from 
here.

Reasoning here is that only the cluster administrator should need to know 
what's on the slave command line.



src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
https://reviews.apache.org/r/28920/#comment112090

please break these out and give a separate error message



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112092

kick the first arg onto the next line, and use +4 indent for all (so a 
method rename doesn't move these out of alignment).



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112091

blank line above



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112095

Can you include a comment explaining why this is necessary?  Will be good 
for posterity.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112100

Prefer inserting line breaks to keep the longest continuous statement 
unbroken.  In this case, i would break after the `=` and put the rest on one 
line.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112096

blank line before



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112097

blank line before



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112098

blank line before



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment112101

line break after the paren, +4

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
  docs/deploying-aurora-scheduler.md, line 155
  https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155
 
  From reading the code, it appears this additional path is needed for 
  them both to be available in the container.  If so, would it be easier to 
  just accept an arbitrary number of additional assets to copy into the 
  sandbox?  I would find that more generalized, and easier to understand.
  
  If you go with the above, i _think_ you can also safely nuke the extra 
  args plumbing.
 
 Steve Niemitz wrote:
 I think we'll still need extra args (its really useful right now to pass 
 ZK config in), but I like the idea of an arbitrary set of resources.  The 
 only trouble here would be figuring out which is the one to run and getting 
 the symlinks right.  Let's talk about this more.
 
 Bill Farner wrote:
 Wouldn't the extra args just be solved with the shim script?  I tell the 
 scheduler to copy `my_executor.sh` and `executor.pex` into the container, and 
 `my_executor.sh` contains the extra args.  I like this as a generalized 
 solution to augmenting default executor behavior, since it avoids feature 
 creep on our side just to save people the shim script.
 
 Steve Niemitz wrote:
 Correct, it would.  My goal with this (which I'm happy to discuss more) 
 was to make the wrapper script unnecessary in normal cases.  For us we just 
 need to pass in the ZK config for the announcer, and I feel like this is the 
 typical case.  Let me simmer on this one for a little bit.
 
 Kevin Sweeney wrote:
 I'd like to entirely avoid the need for a shim script by configuring the 
 scheduler. The rationale being I would like to one day be able to host 
 executor builds on a public artifact store (that a user doesn't necessarily 
 have write access to) with all aurora-specific configuration happening on the 
 scheduler.
 
 In that world the slaves would only need to know about the master, and 
 site-specific executor settings could be configured on the scheduler like so:
 ```
 -thermos_executor_path=https://some.public.url/thermos-0.7.0.pex
 -thermos_executor_flags='--announcer-enable --announcer-ensemble=zk://... 
 ...'
 ```
 
 Bill Farner wrote:
  The rationale being I would like to one day be able to host executor 
 builds on a public artifact store (that a user doesn't necessarily have write 
 access to)
 
 Is this not true today?  It's not clear to me how a shim script implies 
 write access.  On the other hand, i find a host-local shim script useful for 
 things like controlled deployment of new arguments.
 
 Bill Farner wrote:
 Circling back from discussion in IRC - we will proceed with two 
 additioanl command line arguments to the scheduler: one that specifies 
 arguments to the executor, and one that lists additional executor resources 
 required by the executor.  Please reply if you (anyone) disagree.

Just to make sure we're all on the same page, does that mean running with a 
wrapper looks like this?

   -thermos_executor_path=/some/path/to/wrapper.sh
   -thermos_executor_resources=/some/other/path/to/thermos_executor.pex
  
And running without looks like this:
   -thermos_executor_path=/some/other/path/to/thermos_executor.pex
   (no -thermos_executor_resources)

If so I'm on board and will have a change set coming soon.  One thing that 
comes from this however is now each sandbox will have a ~50meg pex file (the 
executor) copied into it.  This probably isn't a huge deal, but I'd just like 
to point it out.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---


On Jan. 9, 2015, 7:34 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 9, 2015, 7:34 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 12, 2015, 7:01 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Changes
---

Changes to thermos_executor_path / wrapper / args to reflect latest code review 
comments.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
  examples/vagrant/provision-dev-cluster.sh 
490a8194424d4adddc2d9b3a7a7f0e579bea6c06 
  examples/vagrant/upstart/aurora-scheduler.conf 
788ec254270bca074dae91829c7f4fccdc8f8bb0 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67682
---


Master (a9e0d34) is red with this patch.
  ./build-support/jenkins/build.sh

:assemble
:compileJmhJavawarning: Supported source version 'RELEASE_6' from annotation 
processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
'1.7'
1 warning

:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh
:jsHint
:checkstyleMain
:compileTestJava
:processTestResources
:testClasses
:checkstyleTest
:findbugsJmh
:findbugsMain
:findbugsTest
:licenseJmh UP-TO-DATE
:licenseMain UP-TO-DATE
:licenseTest UP-TO-DATE
:license UP-TO-DATE
:pmdMain
:test
:jacocoTestReport
Coverage report generated: 
file:///x1/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/jacoco/test/html/index.html
:analyzeReport
Instruction coverage of 0.8924773994098084 exceeds minimum coverage of 0.89.
:analyzeReport FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':analyzeReport'.
 Branch coverage is 0.8345286885245902, but must be greater than 0.835

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output.

BUILD FAILED

Total time: 3 mins 44.607 secs


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 12, 2015, 7:01 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 12, 2015, 7:01 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
   examples/vagrant/provision-dev-cluster.sh 
 490a8194424d4adddc2d9b3a7a7f0e579bea6c06 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67702
---


Master (5ce076b) is red with this patch.
  ./build-support/jenkins/build.sh


Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.lang-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.log
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.log-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.options
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.options-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.process
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.process-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.string
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.string-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.threading
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.threading-0.3.1-py2.7-nspkg.pth
  Running setup.py install for twitter.common.util
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/__init__.py
 (namespace package)
Skipping installation of 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter/common/__init__.py
 (namespace package)

Installing 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/twitter.common.util-0.3.1-py2.7-nspkg.pth
  Found existing installation: setuptools 3.6
Uninstalling setuptools:
  Successfully uninstalled setuptools
Successfully installed Markdown Pygments ansicolors cov-core coverage lockfile 
pantsbuild.pants pex psutil py pystache pytest pytest-cov python-daemon 
requests twitter.common.collections twitter.common.config 
twitter.common.confluence twitter.common.contextutil twitter.common.decorators 
twitter.common.dirutil twitter.common.lang twitter.common.log 
twitter.common.options twitter.common.process twitter.common.string 
twitter.common.threading twitter.common.util setuptools
Cleaning up...

Exception message: Failed to fetch 
https://pypi.python.org/packages/source/w/wheel/wheel-0.23.0.tar.gz#md5=273374d28127b821d61cdfe56e7c35dd
 to /tmp/user/10021/tmp05vcIO: Could not reach 
https://pypi.python.org/packages/source/w/wheel/wheel-0.23.0.tar.gz#md5=273374d28127b821d61cdfe56e7c35dd
 within deadline.



I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 12, 2015, 8:32 p.m., Steve Niemitz wrote:

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 12, 2015, 9:40 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Changes
---

@ReviewBot retry


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
  examples/vagrant/provision-dev-cluster.sh 
490a8194424d4adddc2d9b3a7a7f0e579bea6c06 
  examples/vagrant/upstart/aurora-scheduler.conf 
788ec254270bca074dae91829c7f4fccdc8f8bb0 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  
src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
 dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67722
---


Master (5ce076b) is red with this patch.
  ./build-support/jenkins/build.sh

  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_builder.py,
 line 43, in build
debug=debug).run()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 88, in run
rv = self._run_tests([target], stdout, stderr)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 316, in _run_tests
with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File /usr/lib/python2.7/contextlib.py, line 17, in __enter__
return self.gen.next()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/test_builder.py,
 line 295, in _test_runner
builder = chroot.dump()
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/python_chroot.py,
 line 191, in dump
conn_timeout=self._conn_timeout)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pants/backend/python/resolver.py,
 line 100, in resolve_multi
platform=platform)
  File 
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/pants.venv/lib/python2.7/site-packages/pex/resolver.py,
 line 101, in resolve
raise Unsatisfiable('Cannot satisfy requirements: %s' % 
requirement_set[requirement.key])
Unsatisfiable: Cannot satisfy requirements: [PythonRequirement(pytest-timeout)]


Usage: 
  pants build (options) [spec] (build args)
  pants build (options) [spec]... -- (build args)

Options:
  --version show program's version number and exit
  --no-pantsrc  Specifies that pantsrc files should be ignored.
  --log-exitLog an exit message on success or failure.
  -t CONN_TIMEOUT, --timeout=CONN_TIMEOUT
Number of seconds to wait for http connections.
  -i INTERPRETERS, --interpreter=INTERPRETERS
Constrain what Python interpreters to use.  Uses
Requirement format from pkg_resources, e.g.
'CPython=2.6,3' or 'PyPy'. By default, no
constraints are used.  Multiple constraints may be
added.  They will be ORed together.
  -v, --verbose Show verbose output.
  -f, --fastRun tests in a single chroot.

Builds the specified Python target(s). Use ./pants goal for JVM and other
targets.


I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 12, 2015, 9:40 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 12, 2015, 9:40 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 490a8194424d4adddc2d9b3a7a7f0e579bea6c06 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-12 Thread Bill Farner



 On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
  docs/deploying-aurora-scheduler.md, line 155
  https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155
 
  From reading the code, it appears this additional path is needed for 
  them both to be available in the container.  If so, would it be easier to 
  just accept an arbitrary number of additional assets to copy into the 
  sandbox?  I would find that more generalized, and easier to understand.
  
  If you go with the above, i _think_ you can also safely nuke the extra 
  args plumbing.
 
 Steve Niemitz wrote:
 I think we'll still need extra args (its really useful right now to pass 
 ZK config in), but I like the idea of an arbitrary set of resources.  The 
 only trouble here would be figuring out which is the one to run and getting 
 the symlinks right.  Let's talk about this more.
 
 Bill Farner wrote:
 Wouldn't the extra args just be solved with the shim script?  I tell the 
 scheduler to copy `my_executor.sh` and `executor.pex` into the container, and 
 `my_executor.sh` contains the extra args.  I like this as a generalized 
 solution to augmenting default executor behavior, since it avoids feature 
 creep on our side just to save people the shim script.
 
 Steve Niemitz wrote:
 Correct, it would.  My goal with this (which I'm happy to discuss more) 
 was to make the wrapper script unnecessary in normal cases.  For us we just 
 need to pass in the ZK config for the announcer, and I feel like this is the 
 typical case.  Let me simmer on this one for a little bit.
 
 Kevin Sweeney wrote:
 I'd like to entirely avoid the need for a shim script by configuring the 
 scheduler. The rationale being I would like to one day be able to host 
 executor builds on a public artifact store (that a user doesn't necessarily 
 have write access to) with all aurora-specific configuration happening on the 
 scheduler.
 
 In that world the slaves would only need to know about the master, and 
 site-specific executor settings could be configured on the scheduler like so:
 ```
 -thermos_executor_path=https://some.public.url/thermos-0.7.0.pex
 -thermos_executor_flags='--announcer-enable --announcer-ensemble=zk://... 
 ...'
 ```
 
 Bill Farner wrote:
  The rationale being I would like to one day be able to host executor 
 builds on a public artifact store (that a user doesn't necessarily have write 
 access to)
 
 Is this not true today?  It's not clear to me how a shim script implies 
 write access.  On the other hand, i find a host-local shim script useful for 
 things like controlled deployment of new arguments.
 
 Bill Farner wrote:
 Circling back from discussion in IRC - we will proceed with two 
 additioanl command line arguments to the scheduler: one that specifies 
 arguments to the executor, and one that lists additional executor resources 
 required by the executor.  Please reply if you (anyone) disagree.
 
 Steve Niemitz wrote:
 Just to make sure we're all on the same page, does that mean running with 
 a wrapper looks like this?
 
-thermos_executor_path=/some/path/to/wrapper.sh
-thermos_executor_resources=/some/other/path/to/thermos_executor.pex
   
 And running without looks like this:
-thermos_executor_path=/some/other/path/to/thermos_executor.pex
(no -thermos_executor_resources)
 
 If so I'm on board and will have a change set coming soon.  One thing 
 that comes from this however is now each sandbox will have a ~50meg pex file 
 (the executor) copied into it.  This probably isn't a huge deal, but I'd just 
 like to point it out.

+1, and we will also have the optional argument for additional executor args.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---


On Jan. 12, 2015, 9:40 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 12, 2015, 9:40 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
  docs/deploying-aurora-scheduler.md, line 155
  https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155
 
  From reading the code, it appears this additional path is needed for 
  them both to be available in the container.  If so, would it be easier to 
  just accept an arbitrary number of additional assets to copy into the 
  sandbox?  I would find that more generalized, and easier to understand.
  
  If you go with the above, i _think_ you can also safely nuke the extra 
  args plumbing.
 
 Steve Niemitz wrote:
 I think we'll still need extra args (its really useful right now to pass 
 ZK config in), but I like the idea of an arbitrary set of resources.  The 
 only trouble here would be figuring out which is the one to run and getting 
 the symlinks right.  Let's talk about this more.
 
 Bill Farner wrote:
 Wouldn't the extra args just be solved with the shim script?  I tell the 
 scheduler to copy `my_executor.sh` and `executor.pex` into the container, and 
 `my_executor.sh` contains the extra args.  I like this as a generalized 
 solution to augmenting default executor behavior, since it avoids feature 
 creep on our side just to save people the shim script.
 
 Steve Niemitz wrote:
 Correct, it would.  My goal with this (which I'm happy to discuss more) 
 was to make the wrapper script unnecessary in normal cases.  For us we just 
 need to pass in the ZK config for the announcer, and I feel like this is the 
 typical case.  Let me simmer on this one for a little bit.
 
 Kevin Sweeney wrote:
 I'd like to entirely avoid the need for a shim script by configuring the 
 scheduler. The rationale being I would like to one day be able to host 
 executor builds on a public artifact store (that a user doesn't necessarily 
 have write access to) with all aurora-specific configuration happening on the 
 scheduler.
 
 In that world the slaves would only need to know about the master, and 
 site-specific executor settings could be configured on the scheduler like so:
 ```
 -thermos_executor_path=https://some.public.url/thermos-0.7.0.pex
 -thermos_executor_flags='--announcer-enable --announcer-ensemble=zk://... 
 ...'
 ```
 
 Bill Farner wrote:
  The rationale being I would like to one day be able to host executor 
 builds on a public artifact store (that a user doesn't necessarily have write 
 access to)
 
 Is this not true today?  It's not clear to me how a shim script implies 
 write access.  On the other hand, i find a host-local shim script useful for 
 things like controlled deployment of new arguments.
 
 Bill Farner wrote:
 Circling back from discussion in IRC - we will proceed with two 
 additioanl command line arguments to the scheduler: one that specifies 
 arguments to the executor, and one that lists additional executor resources 
 required by the executor.  Please reply if you (anyone) disagree.
 
 Steve Niemitz wrote:
 Just to make sure we're all on the same page, does that mean running with 
 a wrapper looks like this?
 
-thermos_executor_path=/some/path/to/wrapper.sh
-thermos_executor_resources=/some/other/path/to/thermos_executor.pex
   
 And running without looks like this:
-thermos_executor_path=/some/other/path/to/thermos_executor.pex
(no -thermos_executor_resources)
 
 If so I'm on board and will have a change set coming soon.  One thing 
 that comes from this however is now each sandbox will have a ~50meg pex file 
 (the executor) copied into it.  This probably isn't a huge deal, but I'd just 
 like to point it out.
 
 Bill Farner wrote:
 +1, and we will also have the optional argument for additional executor 
 args.

Great, this should all be incorperated into the new diffs below.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---


On Jan. 12, 2015, 9:40 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 12, 2015, 9:40 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 13, 2015, 1:11 a.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Changes
---

Rebased


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
  examples/vagrant/provision-dev-cluster.sh 
7af4b52a6876268a97630279221bb98d9b04efad 
  examples/vagrant/upstart/aurora-scheduler.conf 
788ec254270bca074dae91829c7f4fccdc8f8bb0 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
f7d8977e42aa56188799400bf8e12a6886fb4a8f 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  
src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
 dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67781
---

Ship it!


Master (c8154b2) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 13, 2015, 1:11 a.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 13, 2015, 1:11 a.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 7af4b52a6876268a97630279221bb98d9b04efad 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 f7d8977e42aa56188799400bf8e12a6886fb4a8f 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67761
---


@ReviewBot retry

- Steve Niemitz


On Jan. 12, 2015, 9:40 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 12, 2015, 9:40 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   config/legacy_untested_classes.txt 33c1d6eb4ea02e01b7002c2c2bae5a3858c8b0e5 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 1e31f21998d02fd69ce0db88e6adb3d32cff67fd 
   examples/vagrant/provision-dev-cluster.sh 
 490a8194424d4adddc2d9b3a7a7f0e579bea6c06 
   examples/vagrant/upstart/aurora-scheduler.conf 
 788ec254270bca074dae91829c7f4fccdc8f8bb0 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
  dc2cb37adf32df0a6e4c7ee2ba776ba9f1f3c2f8 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-09 Thread Bill Farner



 On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
  docs/deploying-aurora-scheduler.md, line 155
  https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155
 
  From reading the code, it appears this additional path is needed for 
  them both to be available in the container.  If so, would it be easier to 
  just accept an arbitrary number of additional assets to copy into the 
  sandbox?  I would find that more generalized, and easier to understand.
  
  If you go with the above, i _think_ you can also safely nuke the extra 
  args plumbing.
 
 Steve Niemitz wrote:
 I think we'll still need extra args (its really useful right now to pass 
 ZK config in), but I like the idea of an arbitrary set of resources.  The 
 only trouble here would be figuring out which is the one to run and getting 
 the symlinks right.  Let's talk about this more.
 
 Bill Farner wrote:
 Wouldn't the extra args just be solved with the shim script?  I tell the 
 scheduler to copy `my_executor.sh` and `executor.pex` into the container, and 
 `my_executor.sh` contains the extra args.  I like this as a generalized 
 solution to augmenting default executor behavior, since it avoids feature 
 creep on our side just to save people the shim script.
 
 Steve Niemitz wrote:
 Correct, it would.  My goal with this (which I'm happy to discuss more) 
 was to make the wrapper script unnecessary in normal cases.  For us we just 
 need to pass in the ZK config for the announcer, and I feel like this is the 
 typical case.  Let me simmer on this one for a little bit.
 
 Kevin Sweeney wrote:
 I'd like to entirely avoid the need for a shim script by configuring the 
 scheduler. The rationale being I would like to one day be able to host 
 executor builds on a public artifact store (that a user doesn't necessarily 
 have write access to) with all aurora-specific configuration happening on the 
 scheduler.
 
 In that world the slaves would only need to know about the master, and 
 site-specific executor settings could be configured on the scheduler like so:
 ```
 -thermos_executor_path=https://some.public.url/thermos-0.7.0.pex
 -thermos_executor_flags='--announcer-enable --announcer-ensemble=zk://... 
 ...'
 ```

 The rationale being I would like to one day be able to host executor builds 
 on a public artifact store (that a user doesn't necessarily have write access 
 to)

Is this not true today?  It's not clear to me how a shim script implies write 
access.  On the other hand, i find a host-local shim script useful for things 
like controlled deployment of new arguments.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---


On Jan. 8, 2015, 8:35 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 8, 2015, 8:35 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-09 Thread Steve Niemitz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 9, 2015, 7:34 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  
src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
 01b03508afac37b5a8f0ec5c3da1460695e1ef59 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-09 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67498
---

Ship it!


Master (74e5471) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 9, 2015, 7:34 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 9, 2015, 7:34 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   
 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
  01b03508afac37b5a8f0ec5c3da1460695e1ef59 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-09 Thread Bill Farner



 On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
  docs/deploying-aurora-scheduler.md, line 155
  https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155
 
  From reading the code, it appears this additional path is needed for 
  them both to be available in the container.  If so, would it be easier to 
  just accept an arbitrary number of additional assets to copy into the 
  sandbox?  I would find that more generalized, and easier to understand.
  
  If you go with the above, i _think_ you can also safely nuke the extra 
  args plumbing.
 
 Steve Niemitz wrote:
 I think we'll still need extra args (its really useful right now to pass 
 ZK config in), but I like the idea of an arbitrary set of resources.  The 
 only trouble here would be figuring out which is the one to run and getting 
 the symlinks right.  Let's talk about this more.
 
 Bill Farner wrote:
 Wouldn't the extra args just be solved with the shim script?  I tell the 
 scheduler to copy `my_executor.sh` and `executor.pex` into the container, and 
 `my_executor.sh` contains the extra args.  I like this as a generalized 
 solution to augmenting default executor behavior, since it avoids feature 
 creep on our side just to save people the shim script.
 
 Steve Niemitz wrote:
 Correct, it would.  My goal with this (which I'm happy to discuss more) 
 was to make the wrapper script unnecessary in normal cases.  For us we just 
 need to pass in the ZK config for the announcer, and I feel like this is the 
 typical case.  Let me simmer on this one for a little bit.
 
 Kevin Sweeney wrote:
 I'd like to entirely avoid the need for a shim script by configuring the 
 scheduler. The rationale being I would like to one day be able to host 
 executor builds on a public artifact store (that a user doesn't necessarily 
 have write access to) with all aurora-specific configuration happening on the 
 scheduler.
 
 In that world the slaves would only need to know about the master, and 
 site-specific executor settings could be configured on the scheduler like so:
 ```
 -thermos_executor_path=https://some.public.url/thermos-0.7.0.pex
 -thermos_executor_flags='--announcer-enable --announcer-ensemble=zk://... 
 ...'
 ```
 
 Bill Farner wrote:
  The rationale being I would like to one day be able to host executor 
 builds on a public artifact store (that a user doesn't necessarily have write 
 access to)
 
 Is this not true today?  It's not clear to me how a shim script implies 
 write access.  On the other hand, i find a host-local shim script useful for 
 things like controlled deployment of new arguments.

Circling back from discussion in IRC - we will proceed with two additioanl 
command line arguments to the scheduler: one that specifies arguments to the 
executor, and one that lists additional executor resources required by the 
executor.  Please reply if you (anyone) disagree.


- Bill


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---


On Jan. 9, 2015, 7:34 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 9, 2015, 7:34 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-08 Thread Kevin Sweeney



 On Jan. 8, 2015, 10:48 a.m., Bill Farner wrote:
  docs/deploying-aurora-scheduler.md, line 155
  https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155
 
  From reading the code, it appears this additional path is needed for 
  them both to be available in the container.  If so, would it be easier to 
  just accept an arbitrary number of additional assets to copy into the 
  sandbox?  I would find that more generalized, and easier to understand.
  
  If you go with the above, i _think_ you can also safely nuke the extra 
  args plumbing.
 
 Steve Niemitz wrote:
 I think we'll still need extra args (its really useful right now to pass 
 ZK config in), but I like the idea of an arbitrary set of resources.  The 
 only trouble here would be figuring out which is the one to run and getting 
 the symlinks right.  Let's talk about this more.
 
 Bill Farner wrote:
 Wouldn't the extra args just be solved with the shim script?  I tell the 
 scheduler to copy `my_executor.sh` and `executor.pex` into the container, and 
 `my_executor.sh` contains the extra args.  I like this as a generalized 
 solution to augmenting default executor behavior, since it avoids feature 
 creep on our side just to save people the shim script.
 
 Steve Niemitz wrote:
 Correct, it would.  My goal with this (which I'm happy to discuss more) 
 was to make the wrapper script unnecessary in normal cases.  For us we just 
 need to pass in the ZK config for the announcer, and I feel like this is the 
 typical case.  Let me simmer on this one for a little bit.

I'd like to entirely avoid the need for a shim script by configuring the 
scheduler. The rationale being I would like to one day be able to host executor 
builds on a public artifact store (that a user doesn't necessarily have write 
access to) with all aurora-specific configuration happening on the scheduler.

In that world the slaves would only need to know about the master, and 
site-specific executor settings could be configured on the scheduler like so:
```
-thermos_executor_path=https://some.public.url/thermos-0.7.0.pex
-thermos_executor_flags='--announcer-enable --announcer-ensemble=zk://... ...'
```


- Kevin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---


On Jan. 8, 2015, 12:35 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 8, 2015, 12:35 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-08 Thread Bill Farner


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---


Great patch overall!  Mostly documentation and style nits, but also opening 
some questions.


api/src/main/thrift/org/apache/aurora/gen/api.thrift
https://reviews.apache.org/r/28920/#comment36

For this doc and others, i found the docs in .md to be more informative, 
please lift them here.  e.g., this was more helpful: `The path inside the 
container where the mount will be created.`.



api/src/main/thrift/org/apache/aurora/gen/api.thrift
https://reviews.apache.org/r/28920/#comment00

This should be camelCase, ditto below.



api/src/main/thrift/org/apache/aurora/gen/api.thrift
https://reviews.apache.org/r/28920/#comment10

This may be my docker ignorance showing, but can you expand on this doc?  
It's not blatantly obvious what this is for and how it differs from 
`container_path`.



api/src/main/thrift/org/apache/aurora/gen/api.thrift
https://reviews.apache.org/r/28920/#comment33

What are the implications of this?  Does it mean a docker image must be 
pre-loaded on the host for it to be used?  Can we cope with the user specifying 
a bad path?  If not, what's the fallout - TASK_FAILED?



api/src/main/thrift/org/apache/aurora/gen/api.thrift
https://reviews.apache.org/r/28920/#comment16

Perhaps this should be called `imageName`?  Without help in the name, i 
could be convinced this was a path or URI.

That said, what is this for?  As a user, can i incorrectly specify/format 
this string, or is it for my own purposes?



api/src/main/thrift/org/apache/aurora/gen/api.thrift
https://reviews.apache.org/r/28920/#comment27

s/A set of z/Z/

As a convention, when type information is already expressed in a 
signature/declaration, tend away from repeating type information in the doc.



docs/configuration-reference.md
https://reviews.apache.org/r/28920/#comment55

rephrase suggestion:

`*Note: the only container type currently supported is docker*`

(italicized, on its own line at the top)



docs/deploying-aurora-scheduler.md
https://reviews.apache.org/r/28920/#comment62

It would be really useful to see a working example of these configuration 
parameters in concert.  I suggest you go ahead and wire it up in the upstart 
configs we have, and link to them from this doc.

Taking it a step further, it would be awesome if this was exercised in 
`./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh`.



docs/deploying-aurora-scheduler.md
https://reviews.apache.org/r/28920/#comment58

linkify docker containerizer to 
http://mesos.apache.org/documentation/latest/docker-containerizer/



docs/deploying-aurora-scheduler.md
https://reviews.apache.org/r/28920/#comment59

s/aurora/mesos/?  IIUC it's the slave that does the copy.



docs/deploying-aurora-scheduler.md
https://reviews.apache.org/r/28920/#comment84

From reading the code, it appears this additional path is needed for them 
both to be available in the container.  If so, would it be easier to just 
accept an arbitrary number of additional assets to copy into the sandbox?  I 
would find that more generalized, and easier to understand.

If you go with the above, i _think_ you can also safely nuke the extra args 
plumbing.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment96

Style nit: we usually leave one blank line between a wrapped method 
signature and the body, for better visual separation.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment92

I applied your patch and removed these PMD exclusions without any issue.  
Are they needed?

As a general practice - we avoid decorating the code with hints to code 
quality checkers.  This could vary from making the code appease the checker to 
disabling the rule.



src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
https://reviews.apache.org/r/28920/#comment82

It's good form for this type of sanitization to happen here, but at minimum 
the same sanitization must be done in `ConfigurationManager` to give the user a 
good message and avoid accepting the bad config.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment99

Declare these as Strings, do the Optional.of(X) where used.



src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java
https://reviews.apache.org/r/28920/#comment97

requireNonNull



src/main/python/apache/aurora/executor/bin/thermos_executor_main.py
https://reviews.apache.org/r/28920/#comment111201

typo: specified



src/main/resources/scheduler/assets/js/services.js

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-08 Thread Steve Niemitz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 8, 2015, 8:35 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
08ba1cdf88b712de22c26c04443079282db59ef9 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-08 Thread Steve Niemitz

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
api/src/main/thrift/org/apache/aurora/gen/api.thrift, line 207
https://reviews.apache.org/r/28920/diff/7/?file=808124#file808124line207

What are the implications of this? Does it mean a docker image must be
pre-loaded on the host for it to be used? Can we cope with the user
specifying a bad path? If not, what's the fallout - TASK_FAILED?

Steve Niemitz wrote:
The Volume options are for mounting paths on the host into the docker
container, and correspond to the -v flag of docker run
(https://docs.docker.com/userguide/dockervolumes/#mount-a-host-directory-as-a-data-volume).
If the path is bad, it will fail but continue to run the container. This
is goverened by mesos, and I actually have some plans to enhance their docker
integration.

Bill Farner wrote:
Awesome, please include that link here and in the .md.

Question remains about whether the image must be pre-loaded on the
machine.

Will do. I'll document it as well but the image doesn't need to be on the
machine, Mesos will pull it if it doesn't exist.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
docs/deploying-aurora-scheduler.md, line 155
https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155

From reading the code, it appears this additional path is needed for
them both to be available in the container. If so, would it be easier to
just accept an arbitrary number of additional assets to copy into the
sandbox? I would find that more generalized, and easier to understand.

If you go with the above, i _think_ you can also safely nuke the extra
args plumbing.

Steve Niemitz wrote:
I think we'll still need extra args (its really useful right now to pass
ZK config in), but I like the idea of an arbitrary set of resources. The
only trouble here would be figuring out which is the one to run and getting
the symlinks right. Let's talk about this more.

Bill Farner wrote:
Wouldn't the extra args just be solved with the shim script? I tell the
scheduler to copy `my_executor.sh` and `executor.pex` into the container, and
`my_executor.sh` contains the extra args. I like this as a generalized
solution to augmenting default executor behavior, since it avoids feature
creep on our side just to save people the shim script.

Correct, it would. My goal with this (which I'm happy to discuss more) was to
make the wrapper script unnecessary in normal cases. For us we just need to
pass in the ZK config for the announcer, and I feel like this is the typical
case. Let me simmer on this one for a little bit.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, line 109
https://reviews.apache.org/r/28920/diff/7/?file=808132#file808132line109

It's good form for this type of sanitization to happen here, but at
minimum the same sanitization must be done in `ConfigurationManager` to
give the user a good message and avoid accepting the bad config.

Steve Niemitz wrote:
I do a similar check in SchedulerMain.java, ~line 211. Should I move the
check into ConfigurationManager? ExecutorSettings.ctor checks as well.

Bill Farner wrote:
Doh, i left this comment in the wrong place. This should have been a
request to sanitize the incoming thrift fields.

Ah that makes more sense! I'll do so.

- Steve

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---

On Jan. 8, 2015, 8:35 p.m., Steve Niemitz wrote:

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 8, 2015, 8:35 p.m.)

Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.

Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633

Repository: aurora

Description
---

This change adds support for launching docker containers through aurora.
These changes are based off of the discussion in
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the
executor inside docker containers. A sample aurora file is in
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`,
the example upstart config in examples/vagrant/upstart has been updated to
reflect this.

More information is in subsequent review request comments.

Diffs
-

Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc
api/src/main/thrift/org/apache/aurora/gen/api.thrift
08ba1cdf88b712de22c26c04443079282db59ef9

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-08 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67289
---

Ship it!


Master (4053924) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 8, 2015, 8:35 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 8, 2015, 8:35 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 08ba1cdf88b712de22c26c04443079282db59ef9 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh b7ea41719a8f41bb23d0254e732926d89399c77c 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-08 Thread Bill Farner

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, line 109
https://reviews.apache.org/r/28920/diff/7/?file=808132#file808132line109

Steve Niemitz wrote:
I do a similar check in SchedulerMain.java, ~line 211. Should I move the
check into ConfigurationManager? ExecutorSettings.ctor checks as well.

Doh, i left this comment in the wrong place. This should have been a request
to sanitize the incoming thrift fields.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
api/src/main/thrift/org/apache/aurora/gen/api.thrift, line 207
https://reviews.apache.org/r/28920/diff/7/?file=808124#file808124line207

Awesome, please include that link here and in the .md.

Question remains about whether the image must be pre-loaded on the machine.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
api/src/main/thrift/org/apache/aurora/gen/api.thrift, line 215
https://reviews.apache.org/r/28920/diff/7/?file=808124#file808124line215

Perhaps this should be called `imageName`? Without help in the name, i
could be convinced this was a path or URI.

That said, what is this for? As a user, can i incorrectly
specify/format this string, or is it for my own purposes?

Steve Niemitz wrote:
I tried to avoid a name with name in it to avoid confusion with other
name fields that are purely for ID reasons (ok that sentence was a
mouthful). I can enhance the docs though to be more clear that it expects a
docker image name and not a URI/etc.

This field is actually the docker container (image) that will be run.

Ah, in that case i agree.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
docs/deploying-aurora-scheduler.md, line 153
https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line153

s/aurora/mesos/? IIUC it's the slave that does the copy.

Steve Niemitz wrote:
how about s/aurora will/aurora will configure mesos to/?

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
docs/deploying-aurora-scheduler.md, line 155
https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155

If you go with the above, i _think_ you can also safely nuke the extra
args plumbing.

Wouldn't the extra args just be solved with the shim script? I tell the
scheduler to copy `my_executor.sh` and `executor.pex` into the container, and
`my_executor.sh` contains the extra args. I like this as a generalized
solution to augmenting default executor behavior, since it avoids feature creep
on our side just to save people the shim script.

- Bill

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---

On Jan. 8, 2015, 8:35 p.m., Steve Niemitz wrote:

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 8, 2015, 8:35 p.m.)

Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.

Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633

Repository: aurora

Description
---

This change adds support for launching docker containers through aurora.
These changes are based off of the discussion in

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-08 Thread Steve Niemitz

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
api/src/main/thrift/org/apache/aurora/gen/api.thrift, line 207
https://reviews.apache.org/r/28920/diff/7/?file=808124#file808124line207

The Volume options are for mounting paths on the host into the docker
container, and correspond to the -v flag of docker run
(https://docs.docker.com/userguide/dockervolumes/#mount-a-host-directory-as-a-data-volume).
If the path is bad, it will fail but continue to run the container. This is
goverened by mesos, and I actually have some plans to enhance their docker
integration.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
api/src/main/thrift/org/apache/aurora/gen/api.thrift, line 215
https://reviews.apache.org/r/28920/diff/7/?file=808124#file808124line215

Perhaps this should be called `imageName`? Without help in the name, i
could be convinced this was a path or URI.

That said, what is this for? As a user, can i incorrectly
specify/format this string, or is it for my own purposes?

I tried to avoid a name with name in it to avoid confusion with other name
fields that are purely for ID reasons (ok that sentence was a mouthful). I can
enhance the docs though to be more clear that it expects a docker image name
and not a URI/etc.

This field is actually the docker container (image) that will be run.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
docs/deploying-aurora-scheduler.md, line 148
https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line148

It would be really useful to see a working example of these
configuration parameters in concert. I suggest you go ahead and wire it up
in the upstart configs we have, and link to them from this doc.

Taking it a step further, it would be awesome if this was exercised in
`./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh`.

Totally agree, I actually have a jira ticket (in our own jira) to do just this.
I assume you mean the vagrant upstart configs?

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
docs/deploying-aurora-scheduler.md, line 153
https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line153

s/aurora/mesos/? IIUC it's the slave that does the copy.

how about s/aurora will/aurora will configure mesos to/?

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
docs/deploying-aurora-scheduler.md, line 155
https://reviews.apache.org/r/28920/diff/7/?file=808126#file808126line155

If you go with the above, i _think_ you can also safely nuke the extra
args plumbing.

I think we'll still need extra args (its really useful right now to pass ZK
config in), but I like the idea of an arbitrary set of resources. The only
trouble here would be figuring out which is the one to run and getting the
symlinks right. Let's talk about this more.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, line 108
https://reviews.apache.org/r/28920/diff/7/?file=808132#file808132line108

I applied your patch and removed these PMD exclusions without any
issue. Are they needed?

As a general practice - we avoid decorating the code with hints to code
quality checkers. This could vary from making the code appease the checker
to disabling the rule.

Ah, so this used to be a != null check that PMD complained about, but since I
refactored it recently to use Optional PMD stopped complaining. I'll remove
these hits.

On Jan. 8, 2015, 6:48 p.m., Bill Farner wrote:
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, line 109
https://reviews.apache.org/r/28920/diff/7/?file=808132#file808132line109

I do a similar check in SchedulerMain.java, ~line 211. Should I move the check
into ConfigurationManager? ExecutorSettings.ctor checks as well.

- Steve

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review67226
---

On Jan. 6, 2015, 11:32 p.m., Steve Niemitz wrote:

---
This is an automatically

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-06 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review66890
---


This patch does not apply cleanly on master (c1174a7), do you need to rebase?

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 5, 2015, 11:53 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 5, 2015, 11:53 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 5665c69cd7b49c3fd7345074c9f16a3b224496ab 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
 ead9d28100673440168a32d114ecaa15874978a6 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-06 Thread Steve Niemitz


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 6, 2015, 11:32 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Changes
---

Some refactoring to use Optional more.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
5665c69cd7b49c3fd7345074c9f16a3b224496ab 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-06 Thread Aurora ReviewBot


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review66950
---

Ship it!


Master (8c49029) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing @ReviewBot 
retry

- Aurora ReviewBot


On Jan. 6, 2015, 11:32 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 6, 2015, 11:32 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 5665c69cd7b49c3fd7345074c9f16a3b224496ab 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/main/resources/scheduler/assets/configSummary.html 
 28878908b0c9381e366b71a3135dfc28c542a890 
   src/main/resources/scheduler/assets/js/services.js 
 b744f375411e09b7f776e4a05ee5961227143439 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-05 Thread Jay Buffington



 On Dec. 13, 2014, 1:59 a.m., Jay Buffington wrote:
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java, line 
  279
  https://reviews.apache.org/r/28920/diff/2/?file=789365#file789365line279
 
  I would do away with this TaskConfig hasProcesses field.  You should 
  just use config.isSetExecutorConfig() (and of course not set executor 
  config in the python client when there are no processes.)
 
 Steve Niemitz wrote:
 I'm going to remove support for running the container directly w/o the 
 executor for now.  Without the job registering with the observer, aurora 
 thinks the job is lost and will GC it or abandon it.  We can add this as a 
 to-do in the future.

I'm +1 on this.  I think running without the executor is of dubious value 
anyway.  I hadn't considered the issue with the GC.  Thanks for pointing this 
out!


- Jay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review65000
---


On Dec. 26, 2014, 9:05 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Dec. 26, 2014, 9:05 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample script is in 
 examples/jobs/docker, as well as an example aurora file.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 The thermos root path defaults to /var/run/thermos, however if a different 
 path is used, it must be passed to the scheduler via 
 `--thermos_observer_root=some path`
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 5665c69cd7b49c3fd7345074c9f16a3b224496ab 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
 ead9d28100673440168a32d114ecaa15874978a6 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora



 On Jan. 5, 2015, 8:41 p.m., Joshua Cohen wrote:
  This is 99% just style nits. Unfortunately our Java styleguide isn't 
  published, but I'm working on rectifying that!

Thanks for all the feedback, new patch set coming up!


 On Jan. 5, 2015, 8:41 p.m., Joshua Cohen wrote:
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java, lines 
  223-224
  https://reviews.apache.org/r/28920/diff/4/?file=801606#file801606line223
 
  Do we need to plumb the wrapper path all the way through to the 
  CommandUtil? What if instead of passing both along we figure out the 
  correct path to use here at start up and passed that along directly?
  
  I.e. we could do something like...
  
 OptionalString executorPath = 
  Optional.of(THERMOS_EXECUTOR_PATH.get()).or(Optional.of(THERMOS_EXECUTOR_WRAPPER_PATH.get()));
 
 if (!executorPath.isPresent()) {
   throw new IllegalStateException(...);
 }
 
 bind(ExecutorSettings.class).toInstance(new ExecutorSettings(
 executorPath.get(),
 ...));

We need both paths in the ExecutorSettings so they can both get added to the 
CommandInfo as resources (for docker).


 On Jan. 5, 2015, 8:41 p.m., Joshua Cohen wrote:
  src/test/python/apache/aurora/executor/test_thermos_executor.py, line 193
  https://reviews.apache.org/r/28920/diff/4/?file=801621#file801621line193
 
  This seems like a change in semantics... the sandbox_provider 
  previously was expected to be a factory function that returned the sandbox, 
  now you're passing in the sandbox itself? Why the change?

The UserOverrideDirectorySandboxProvider has a constructor which takes 
arguments, so the provider can no longer be parameterless.  I feel like this is 
more true to a factory pattern anyways, instead of passing in a factory 
factory we're now just passing in a factory.


- Steve


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review66702
---


On Jan. 5, 2015, 8:25 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Jan. 5, 2015, 8:25 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample aurora file is in 
 examples/jobs/docker.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 More information is in subsequent review request comments.
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 5665c69cd7b49c3fd7345074c9f16a3b224496ab 
   docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
   docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
 ead9d28100673440168a32d114ecaa15874978a6 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 5, 2015, 11:53 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
5665c69cd7b49c3fd7345074c9f16a3b224496ab 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
ead9d28100673440168a32d114ecaa15874978a6 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 5, 2015, 7:56 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Changes
---

Updated docs and refactored CommandUtil slightly.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample script is in examples/jobs/docker, 
as well as an example aurora file.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

The thermos root path defaults to /var/run/thermos, however if a different path 
is used, it must be passed to the scheduler via `--thermos_observer_root=some 
path`


Diffs (updated)
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
5665c69cd7b49c3fd7345074c9f16a3b224496ab 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
ead9d28100673440168a32d114ecaa15874978a6 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/
---

(Updated Jan. 5, 2015, 8:25 p.m.)


Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.


Bugs: AURORA-633
https://issues.apache.org/jira/browse/AURORA-633


Repository: aurora


Description (updated)
---

This change adds support for launching docker containers through aurora.  These 
changes are based off of the discussion in 
https://issues.apache.org/jira/browse/AURORA-633

As of now, a special thermos_executor.sh script is needed to launch the 
executor inside docker containers.  A sample aurora file is in 
examples/jobs/docker.

In addition, mesos-slave must be run with `--containerizers=docker,mesos`, the 
example upstart config in examples/vagrant/upstart has been updated to reflect 
this.

More information is in subsequent review request comments.


Diffs
-

  Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
  api/src/main/thrift/org/apache/aurora/gen/api.thrift 
5665c69cd7b49c3fd7345074c9f16a3b224496ab 
  docs/configuration-reference.md f3cb257206a194b82fd2045dc20456ee832dbcea 
  docs/deploying-aurora-scheduler.md 711ae7eda07c2c1735601c265c06a88c1862cce7 
  examples/jobs/docker/hello_docker.aurora PRE-CREATION 
  examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
  examples/vagrant/upstart/mesos-slave.conf 
512ce7ecf34042ed68dda55efb2dd0415f8469db 
  src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
  src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
5226e3d1b303b1773a057078f2911c5ec2aa97f5 
  src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
ead9d28100673440168a32d114ecaa15874978a6 
  src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
d885b224ec5a1d529347d84e03ba98ab6734a126 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
  src/main/python/apache/aurora/config/thrift.py 
ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
  src/main/python/apache/aurora/executor/aurora_executor.py 
636b23d30a897b557eb8c3f8733c90b23cb807ef 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
  src/main/python/apache/aurora/executor/common/sandbox.py 
f47a32b3fefb4a89940b1ddc473b8316ac00df12 
  src/main/python/apache/aurora/executor/thermos_task_runner.py 
5e4bd65537d186459003c0b9434f1b769e04f448 
  src/main/python/apache/thermos/config/schema_base.py 
f9143cc1b83143d6147f59d90c79435d055d0518 
  src/main/python/apache/thermos/core/runner.py 
8aac6b50c66080abbb5308b367e9f74c487f42e3 
  src/main/resources/scheduler/assets/configSummary.html 
28878908b0c9381e366b71a3135dfc28c542a890 
  src/main/resources/scheduler/assets/js/services.js 
b744f375411e09b7f776e4a05ee5961227143439 
  src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
5e54364a49a208bd5f19b9649633dc8feca591e9 
  src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
876e173ccbac04e4a06a245648c7c6af15eaaa92 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
  src/test/python/apache/aurora/executor/test_thermos_executor.py 
503e62f4cac872b14f6985b5bccc3e4dfcf81789 

Diff: https://reviews.apache.org/r/28920/diff/


Testing
---


Thanks,

Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora

2015-01-05 Thread Kevin Sweeney


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28920/#review66692
---



src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java
https://reviews.apache.org/r/28920/#comment110272

This can't be made private due to the way @Timed works (for more 
information see https://github.com/google/guice/wiki/AOP).


- Kevin Sweeney


On Dec. 26, 2014, 1:05 p.m., Steve Niemitz wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/28920/
 ---
 
 (Updated Dec. 26, 2014, 1:05 p.m.)
 
 
 Review request for Aurora, Jay Buffington, Kevin Sweeney, and Bill Farner.
 
 
 Bugs: AURORA-633
 https://issues.apache.org/jira/browse/AURORA-633
 
 
 Repository: aurora
 
 
 Description
 ---
 
 This change adds support for launching docker containers through aurora.  
 These changes are based off of the discussion in 
 https://issues.apache.org/jira/browse/AURORA-633
 
 As of now, a special thermos_executor.sh script is needed to launch the 
 executor inside docker containers.  A sample script is in 
 examples/jobs/docker, as well as an example aurora file.
 
 In addition, mesos-slave must be run with `--containerizers=docker,mesos`, 
 the example upstart config in examples/vagrant/upstart has been updated to 
 reflect this.
 
 The thermos root path defaults to /var/run/thermos, however if a different 
 path is used, it must be passed to the scheduler via 
 `--thermos_observer_root=some path`
 
 
 Diffs
 -
 
   Vagrantfile f8b7db8eebdc6a10989de3bc9a2c3e89ce17f5fc 
   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
 5665c69cd7b49c3fd7345074c9f16a3b224496ab 
   examples/jobs/docker/hello_docker.aurora PRE-CREATION 
   examples/vagrant/aurorabuild.sh 69983d0140b76c6869cd04e55d760f3e3a1e4262 
   examples/vagrant/upstart/mesos-slave.conf 
 512ce7ecf34042ed68dda55efb2dd0415f8469db 
   src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 
 72c7545e7f16549f6a9ccb5fb74a06f154a7ea94 
   src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 
 5226e3d1b303b1773a057078f2911c5ec2aa97f5 
   src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java 
 ead9d28100673440168a32d114ecaa15874978a6 
   src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 
 d885b224ec5a1d529347d84e03ba98ab6734a126 
   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
 5bf283062c9d119ff91ed45da8b236e36d0fc9aa 
   src/main/python/apache/aurora/config/thrift.py 
 ba94ac3c0cbaf3c91eb1a1d86a244ed6fa3b649c 
   src/main/python/apache/aurora/executor/aurora_executor.py 
 636b23d30a897b557eb8c3f8733c90b23cb807ef 
   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 
 9df9b4b79c0c7d29c5088409bf15c0d32a621df0 
   src/main/python/apache/aurora/executor/common/sandbox.py 
 f47a32b3fefb4a89940b1ddc473b8316ac00df12 
   src/main/python/apache/aurora/executor/thermos_task_runner.py 
 5e4bd65537d186459003c0b9434f1b769e04f448 
   src/main/python/apache/thermos/config/schema_base.py 
 f9143cc1b83143d6147f59d90c79435d055d0518 
   src/main/python/apache/thermos/core/runner.py 
 8aac6b50c66080abbb5308b367e9f74c487f42e3 
   src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 
 5e54364a49a208bd5f19b9649633dc8feca591e9 
   src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java 
 876e173ccbac04e4a06a245648c7c6af15eaaa92 
   
 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
 ddcb511d108220ab5e4efcf3496458f7ab4a20c2 
   src/test/python/apache/aurora/executor/test_thermos_executor.py 
 503e62f4cac872b14f6985b5bccc3e4dfcf81789 
 
 Diff: https://reviews.apache.org/r/28920/diff/
 
 
 Testing
 ---
 
 
 Thanks,
 
 Steve Niemitz

Re: Review Request 28920: Add support for docker containers to aurora