----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47853/ -----------------------------------------------------------
Review request for Aurora, Maxim Khutornenko and Stephan Erb. Repository: aurora Description ------- This changes the approach to launching tasks with filesystem images in the unified containerizer. Instead of adding an `Image` to the `MesosContainer`, we instead add the task filesystem as a `Volume` with an associated image. This image is mounted in the mesos directory under the `taskfs` path. The executor, on start up does the following: 1. Creates user/group under the taskfs root. 2. `pivot_root`s into the taskfs, while bind mounting the sandbox under that root as well as mounting procfs. 3. From there, task execution is essentially unchanged minus some slight changes to the environment depending on whether we're running in a pivoted root. Diffs ----- api/src/main/thrift/org/apache/aurora/gen/api.thrift a99889c1f2d9e10825f87ea669532ad78641880f examples/vagrant/upstart/aurora-scheduler.conf 3d9e706de564df5e24cb34265bebc0db1cad11a0 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 3b01801d929dd61ee989495bf38af8f03e9f5ad4 src/main/python/apache/aurora/executor/common/sandbox.py be1deba6219462c9fdaaf07a583851b85fe974bf src/main/python/apache/aurora/executor/thermos_task_runner.py 3896e3841562600379705dbf78a6f62728246348 src/main/python/apache/thermos/core/BUILD 1094664e112cc71af37835f32037e9eb6d047202 src/main/python/apache/thermos/core/process.py 1791b5ff9a36eef7470bef9a6ebbafaf0ab05ca3 src/main/python/apache/thermos/core/runner.py 3ebf86ebd12ed3b68f543d4b9a45615e4681ba7f src/main/python/apache/thermos/runner/thermos_runner.py 0d06e8e2ac78d26ba8f63744853eb5ce3f6aced6 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 58785bfa37ff214f26e9f94d836e6df40e411c3b src/test/python/apache/aurora/executor/common/test_sandbox.py e47d9b8822deb36cb9cfa0554ef89d6cda80f3e9 src/test/python/apache/thermos/core/test_process.py 77f644c09116266ce02479b9a80403aa68767bd6 src/test/sh/org/apache/aurora/e2e/Dockerfile 6fdea3d28760f59235c51c5b6913d2ee0172ef1a src/test/sh/org/apache/aurora/e2e/Dockerfile.netcat PRE-CREATION src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 219c40fb94561f0a390cac16e643bf4332c51aad src/test/sh/org/apache/aurora/e2e/http/http_example_bad_healthcheck.aurora 08553e4f48f137e0455ad07287086311171c06bd src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 8b3a50ba6de992560593987f3db254baa4d29a41 src/test/sh/org/apache/aurora/e2e/run-server.sh PRE-CREATION src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh abe0ca75c6a2c0ace15fce68ad0e5c9aa98193a4 Diff: https://reviews.apache.org/r/47853/diff/ Testing ------- Lots of manual testing, e2e tests, etc. I didn't add much coverage on the thermos side of things because it seemed like this was better served by the e2e tests than by doing a bunch of subprocess.check_call mocking. On the e2e front I created a new Dockerfile that sets up a much slimmer filesystem image that explicitly does not include python to ensure that the executor's filesystem is truly isolated. Thanks, Joshua Cohen
