----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53403/#review154650 -----------------------------------------------------------
Master (13d4861) is red with this patch. ./build-support/jenkins/build.sh Running setup.py bdist_wheel for cov-core: started Running setup.py bdist_wheel for cov-core: finished with status 'done' Stored in directory: /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/86/e1/c2/9ff8cfe9773ce07003f2c2be096e169af4614c2f634671d49b Running setup.py bdist_wheel for pycparser: started Running setup.py bdist_wheel for pycparser: finished with status 'done' Stored in directory: /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/a8/0b/41/dc95621f9d3a0da7bc191b8a71f0e8182ffd3cc5f33ac55005 Running setup.py bdist_wheel for twitter.common.options: started Running setup.py bdist_wheel for twitter.common.options: finished with status 'done' Stored in directory: /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/17/41/80/c4811d8c1c7ca7007e520c3399872fc340f45c3a26a6a23e6a Successfully built pantsbuild.pants twitter.common.collections setproctitle ansicolors pathspec scandir twitter.common.dirutil pystache psutil docutils Markdown Pygments twitter.common.confluence coverage pytest pytest-cov lmdb pywatchman twitter.common.lang twitter.common.log cov-core pycparser twitter.common.options Installing collected packages: twitter.common.lang, twitter.common.collections, setproctitle, setuptools, six, ansicolors, pyparsing, packaging, pathspec, scandir, twitter.common.dirutil, requests, pystache, psutil, pex, docutils, Markdown, Pygments, twitter.common.options, twitter.common.log, twitter.common.confluence, monotonic, fasteners, coverage, py, pytest, cov-core, pytest-cov, futures, pycparser, cffi, lmdb, pywatchman, pantsbuild.pants Found existing installation: setuptools 21.2.1 Uninstalling setuptools-21.2.1: Successfully uninstalled setuptools-21.2.1 Successfully installed Markdown-2.1.1 Pygments-1.4 ansicolors-1.0.2 cffi-1.7.0 cov-core-1.15.0 coverage-3.7.1 docutils-0.12 fasteners-0.14.1 futures-3.0.5 lmdb-0.89 monotonic-1.2 packaging-16.7 pantsbuild.pants-1.3.0.dev0 pathspec-0.3.4 pex-1.1.13 psutil-4.3.0 py-1.4.31 pycparser-2.17 pyparsing-2.1.10 pystache-0.5.3 pytest-2.6.4 pytest-cov-1.8.1 pywatchman-1.3.0 requests-2.5.3 scandir-1.2 setproctitle-1.1.10 setuptools-5.4.1 six-1.10.0 twitter.common.collections-0.3.9 twitter.common.confluence-0.3.9 twitter.common.dirutil-0.3.9 twitter.common.lang-0.3.9 twitter.common.log-0.3.9 twitter.common.options-0.3.9 You are using pip version 8.1.2, however version 9.0.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command. 22:40:31 00:00 [main] (To run a reporting server: ./pants server) 22:40:31 00:00 [setup] 22:40:31 00:00 [parse][32m Executing tasks in goals: compile[0m 22:40:32 00:01 [compile] 22:40:32 00:01 [compile-prep-command] 22:40:32 00:01 [compile] 22:40:32 00:01 [python-eval] 22:40:32 00:01 [pythonstyle] 22:40:32 00:01 [cache] [32m No cached artifacts for 42 targets.[0m[32m Invalidated 42 targets.[0m F841:ERROR src/main/python/apache/thermos/common/process_util.py:065 local variable 'prctl' is assigned to but never used | prctl = libc.prctl FAILURE: 1 Python Style issues found. For import order related issues, please try `./pants fmt.isort <targets>` 22:40:51 00:20 [complete][31m FAILURE[0m I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Nov. 2, 2016, 10:16 p.m., Zameer Manji wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53403/ > ----------------------------------------------------------- > > (Updated Nov. 2, 2016, 10:16 p.m.) > > > Review request for Aurora, Joshua Cohen, Santhosh Kumar Shanmugham, and > Stephan Erb. > > > Bugs: AURORA-1808 > https://issues.apache.org/jira/browse/AURORA-1808 > > > Repository: aurora > > > Description > ------- > > This is a WIP patch showing a possible fix to AURORA-1808. > > # Problem > > Processes can deamonize and escape the supervision of a coordinator. Using > the Docker Containerizer or the Mesos Containerizer with pid isolation means > that the processes will be come reparented to the `sh` process that launches > the executor. For example: > ```` > root@aurora:/# ps xf > PID TTY STAT TIME COMMAND > 48 ? Ss 0:00 /bin/bash > 86 ? R+ 0:00 _ ps xf > 1 ? Ss 0:00 /bin/sh -c ${MESOS_SANDBOX=.}/thermos_executor.pex > --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config > /home/vagrant/aurora/examples/va > 5 ? Sl 0:02 python2.7 /mnt/mesos/sandbox/thermos_executor.pex > --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config > /home/vagrant/aurora/examples/vag > 23 ? S 0:00 _ /usr/local/bin/python2.7 > /mnt/mesos/sandbox/thermos_runner.pex > --task_id=www-data-devel-hello_docker_engine-0-bde5cdc7-8685-46fd-9078-4a86bd5be152 > -- > 29 ? Ss 0:00 _ /usr/local/bin/python2.7 > /mnt/mesos/sandbox/thermos_runner.pex > --task_id=www-data-devel-hello_docker_engine-0-bde5cdc7-8685-46fd-9078-4a86bd5be15 > 32 ? S 0:00 | _ /bin/bash -c while true; do > echo hello world sleep 10 done > 81 ? S 0:00 | _ sleep 10 > 31 ? Ss 0:00 _ /usr/local/bin/python2.7 > /mnt/mesos/sandbox/thermos_runner.pex > --task_id=www-data-devel-hello_docker_engine-0-bde5cdc7-8685-46fd-9078-4a86bd5be15 > 33 ? S 0:00 _ /bin/bash -c while true; do > echo hello world sleep 10 done > 82 ? S 0:00 _ sleep 10 > 47 ? S 0:00 python ./daemon.py > ```` > > # Solution > Ensure processes that escape the supervision of the coordinator reparent to > the runner who can send signals to them on task tear down. We do this by > using the `PR_SET_CHILD_SUBREAPER` flag of `prctl(2)`. > > After this change the process tree looks like: > ```` > root@aurora:/# ps xf > PID TTY STAT TIME COMMAND > 66 ? Ss 0:00 /bin/bash > 70 ? R+ 0:00 _ ps xf > 1 ? Ss 0:00 /bin/sh -c ${MESOS_SANDBOX=.}/thermos_executor.pex > --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config > /home/vagrant/aurora/examples/va > 5 ? Sl 0:02 python2.7 /mnt/mesos/sandbox/thermos_executor.pex > --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config > /home/vagrant/aurora/examples/vag > 23 ? S 0:00 _ /usr/local/bin/python2.7 > /mnt/mesos/sandbox/thermos_runner.pex > --task_id=www-data-devel-hello_docker_engine-0-721406db-00f5-4c0c-915e-1dbc5568b849 > -- > 33 ? Ss 0:00 _ /usr/local/bin/python2.7 > /mnt/mesos/sandbox/thermos_runner.pex > --task_id=www-data-devel-hello_docker_engine-0-721406db-00f5-4c0c-915e-1dbc5568b84 > 40 ? S 0:00 | _ /bin/bash -c while true; do > echo hello world sleep 10 done > 63 ? S 0:00 | _ sleep 10 > 36 ? Ss 0:00 _ /usr/local/bin/python2.7 > /mnt/mesos/sandbox/thermos_runner.pex > --task_id=www-data-devel-hello_docker_engine-0-721406db-00f5-4c0c-915e-1dbc5568b84 > 37 ? S 0:00 | _ /bin/bash -c while true; do > echo hello world sleep 10 done > 62 ? S 0:00 | _ sleep 10 > 55 ? S 0:00 _ python ./daemon.py > ```` > > Now the runner is aware of the reparented procesess can can tear it down > cleanly during teardown. > > Note that the man page for `prctl(2)` says that the processes that set > `PR_SET_CHILD_SUBREAPER` should reap children to get rid of zombies. It is > important to note tht the runner already does this in its run loop via > `TaskRunnerHelper.reap_children()`. This patch has the side effect of > ensuring it will reap all of the children launched via coordinators. > > > Diffs > ----- > > src/main/python/apache/thermos/common/process_util.py > abd2c0ef35858d13971319b0a7436ce2293824ce > src/main/python/apache/thermos/core/helper.py > 68855e1e54ba1cd4456e18a36fb237ce6a468c34 > src/main/python/apache/thermos/core/process.py > 3ec43e2719ef97026f399c4b2aa23002559b3153 > src/main/python/apache/thermos/core/runner.py > 7b9013d11f6ff4172b6b7bf56e62299b0d11c977 > > Diff: https://reviews.apache.org/r/53403/diff/ > > > Testing > ------- > > no automated tests yet. > > Validated behaviour with `ps` and `strace`. > > > Thanks, > > Zameer Manji > >