> On May 4, 2016, 12:19 a.m., Maxim Khutornenko wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java,
> >  lines 76-78
> > <https://reviews.apache.org/r/46835/diff/2/?file=1370290#file1370290line76>
> >
> >     Does this have to be bundled into this RB? We are still using this path 
> > in DEFAULT_CHECKPOINT_ROOT on the thermos side. Perhaps, have a more 
> > thorough cleanup in a separate patch?
> 
> Joshua Cohen wrote:
>     I need to drop the mount for the e2e test to work. I can drop the mount 
> and leave the flag around unused for a later cleanup, but that will require 
> checkstyle suppressions. If you'd prefer I can send out a separate patch that 
> drops everything related to this arg and rebase this on top of it once it 
> lands.
> 
> Maxim Khutornenko wrote:
>     Please, keep that cleanup on your radar.
> 
> Joshua Cohen wrote:
>     I was about to file a ticket for this, but I'm not sure we *want* to 
> clean up this code from the Observer. In theory it's possible someone out 
> there has some long running tasks where they're running a more recent version 
> of the scheduler and the executor, but still have older executors running 
> that are writing to the global checkpoint root. In this case it makes sense 
> for the observer to continue performing lookups in both places.

I filed https://issues.apache.org/jira/browse/AURORA-1689 to track this one way 
or another (if we decide to maintain support we can just resolve as Won't Fix).


- Joshua


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46835/#review131585
-----------------------------------------------------------


On May 4, 2016, 2:36 p.m., Joshua Cohen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46835/
> -----------------------------------------------------------
> 
> (Updated May 4, 2016, 2:36 p.m.)
> 
> 
> Review request for Aurora, John Sirois, Maxim Khutornenko, and Bill Farner.
> 
> 
> Bugs: AURORA-1636, AURORA-1637, AURORA-1638, and AURORA-1639
>     https://issues.apache.org/jira/browse/AURORA-1636
>     https://issues.apache.org/jira/browse/AURORA-1637
>     https://issues.apache.org/jira/browse/AURORA-1638
>     https://issues.apache.org/jira/browse/AURORA-1639
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> A few notes:
> 
> 1. It's not possible to configure Mesos 0.27.x to launch docker tasks due to 
> a bug in parsing the docker_store_dir flag. Fixed here: 
> https://reviews.apache.org/r/43451/ but has not been backported to Mesos 
> 0.27. This means we can only launch tasks that use AppC images until we 
> upgrade our Mesos dependency to 0.28.x. The good news is I've confirmed that 
> launching tasks with Docker images *does* work by using Aurora linked against 
> 0.27.x but running Mesos 0.28.x in Vagrant.
> 1. In order to work around the setuid issues (i.e. task is launched as root, 
> but the executor cannot setuid because the role-user does not exist), I've 
> mounted /etc/passwd and /etc/group into the container and added a new flag, 
> `thermos_run_as_job_role`, to the scheduler. This flag is only used when 
> launching a task with a filesystem image, and causes us to add 
> `--execute-as-user <role from job key>` to the thermos executor commandline.
> 1. The Mesos unified containerizer does not automatically create mount points 
> in the filesystem from the image. It expects the full path to the mount to 
> exist in the image. For /etc/passwd and /etc/groups this is not a problem, 
> but for the announcer acls file it was. I ended up moving the announcer acl 
> file into its own directory and mount that instead. In conjunction with this 
> I also had to modify our http_example Dockerfile to explicitly create that 
> mount point. A case could be made for sticking with the current path and just 
> creating an empty file in the image, I felt that creating an empty directory 
> was slightly less gross. This is tracked by 
> https://issues.apache.org/jira/browse/MESOS-5229.
> 1. The AppC image for end to end tests is created by running 
> [docker2aci](https://github.com/appc/docker2aci) on our http_example docker 
> image. The base box needed to be upgraded to add this utility. I haven't 
> published the new base box yet even though I've updated the Vagrantfile to 
> point to version 6. Once this review has been approved and I'm sure there's 
> no further changes that need to be made I'll publish the base box before 
> committing.
> 
> 
> Diffs
> -----
> 
>   3rdparty/python/requirements.txt 666c4ae487332f01380cfce76f0d97e2c6049c8e 
>   RELEASE-NOTES.md 8d5cbed2c627948c585241a8292a264e3d86120d 
>   Vagrantfile 3f126ee348d0f95d6f159b62280de79f41e87e2e 
>   api/src/main/thrift/org/apache/aurora/gen/api.thrift 
> 38470951e4482753fcada109ab12546a2fb146ce 
>   build-support/packer/build.sh 76197c31c365aa3d8a67049da40b2976c1e25d22 
>   docs/reference/configuration.md 9fcfdfcd9ab793e888ca2bba2035d5122142a5ab 
>   docs/reference/scheduler-configuration.md 
> 5f898a8d180cec6f3c02cb5b01673c56308ebd8a 
>   examples/vagrant/announcer-auth.json  
>   examples/vagrant/mesos_config/etc_mesos-slave/appc_store_dir PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-slave/image_providers PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-slave/image_provisioner_backend 
> PRE-CREATION 
>   examples/vagrant/mesos_config/etc_mesos-slave/isolation PRE-CREATION 
>   examples/vagrant/upstart/aurora-scheduler.conf 
> 084016abc169ed82b7ed00f5d14aea2e0ff38a49 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java
>  e1ce6380e08178e0cd6b1f1651e49c7a4337fb94 
>   
> src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java
>  32f2fa90b21189180e2bcd65a3cebf13f6551646 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 
> b325106c7f45b1ad1657221aaa39e3a428719ab0 
>   src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java 
> 9aadcebf547bd1eb4b4e238507e27ae2b699f473 
>   src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java 
> 9eadf70240738030593a819a4a248588c9d1c35b 
>   src/main/java/org/apache/aurora/scheduler/storage/db/views/DbContainer.java 
> ae97638fa544dd8f8afbaa19b1dd31f5a1dc43d8 
>   
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbTaskConfig.java 
> a7523c4b258030bcfb2e457b083242ffa865a98a 
>   src/main/python/apache/aurora/config/schema/base.py 
> 00be8747d70dbf1cb370f09536588f8602d8fcce 
>   src/main/python/apache/aurora/config/thrift.py 
> 928ca9313b2c2062a322ba80b504a09c55e5377f 
>   src/main/python/apache/aurora/executor/common/sandbox.py 
> 36f1eabedc3ae47b23d9ab2ac0ab7a576ea36fd7 
>   
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.xml
>  cd181bb7ba0b3d725b51b1fe35303afa77294cb9 
>   
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
>  98fe86000c2efa93a415824a680c1e144b0fac11 
>   
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 
> ad397c6924b025f1eefb2bb02a6dc1e1f10ca078 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractCronJobStoreTest.java
>  2343394a34541bc266f74b5da512cda6f8e6b56a 
>   
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 
> af56115a4160e362d41bc9a571f5a1ca623ad199 
>   
> src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java
>  e43ec6cb35cbb454b967238dfb9ce006b21f4fb6 
>   src/test/python/apache/aurora/config/test_thrift.py 
> 7a076f0350ab2967abc6b8b7a2e5da0817926a56 
>   src/test/python/apache/aurora/executor/common/test_sandbox.py 
> bd402fc03c7790eab0198dd48414ad4de138e195 
>   src/test/sh/org/apache/aurora/e2e/Dockerfile 
> b2557b5a20cc369e31bd10ea92462bdb1879add7 
>   src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 
> 2813b6c79e4d44007dde79a10e2c7c9e9c1cecd9 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_bad_healthcheck.aurora 
> 0534c9e589d10c53b834850477f95ad15b50010e 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 
> b33e8f5cd95ce25ba0dc4c08da32783cecf1c44d 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 
> eee6b4c62130567ecd5c32603feae88fce1c13a8 
> 
> Diff: https://reviews.apache.org/r/46835/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build -Pq
> e2e tests with new base box.
> 
> 
> Thanks,
> 
> Joshua Cohen
> 
>

Reply via email to