-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53418/#review154836
-----------------------------------------------------------
Master (171a8f7) is red with this patch.
./build-support/jenkins/build.sh
Running setup.py bdist_wheel for pycparser: finished with status 'done'
Stored in directory:
/home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/a8/0b/41/dc95621f9d3a0da7bc191b8a71f0e8182ffd3cc5f33ac55005
Running setup.py bdist_wheel for twitter.common.options: started
Running setup.py bdist_wheel for twitter.common.options: finished with status
'done'
Stored in directory:
/home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/17/41/80/c4811d8c1c7ca7007e520c3399872fc340f45c3a26a6a23e6a
Successfully built pantsbuild.pants twitter.common.collections setproctitle
ansicolors pathspec scandir twitter.common.dirutil pystache psutil docutils
Markdown Pygments twitter.common.confluence coverage pytest pytest-cov lmdb
pywatchman twitter.common.lang twitter.common.log cov-core pycparser
twitter.common.options
Installing collected packages: twitter.common.lang, twitter.common.collections,
setproctitle, setuptools, six, ansicolors, pyparsing, packaging, pathspec,
scandir, twitter.common.dirutil, requests, pystache, psutil, pex, docutils,
Markdown, Pygments, twitter.common.options, twitter.common.log,
twitter.common.confluence, monotonic, fasteners, coverage, py, pytest,
cov-core, pytest-cov, futures, pycparser, cffi, lmdb, pywatchman,
pantsbuild.pants
Found existing installation: setuptools 21.2.1
Uninstalling setuptools-21.2.1:
Successfully uninstalled setuptools-21.2.1
Successfully installed Markdown-2.1.1 Pygments-1.4 ansicolors-1.0.2 cffi-1.7.0
cov-core-1.15.0 coverage-3.7.1 docutils-0.12 fasteners-0.14.1 futures-3.0.5
lmdb-0.89 monotonic-1.2 packaging-16.7 pantsbuild.pants-1.3.0.dev0
pathspec-0.3.4 pex-1.1.13 psutil-4.3.0 py-1.4.31 pycparser-2.17
pyparsing-2.1.10 pystache-0.5.3 pytest-2.6.4 pytest-cov-1.8.1 pywatchman-1.3.0
requests-2.5.3 scandir-1.2 setproctitle-1.1.10 setuptools-5.4.1 six-1.10.0
twitter.common.collections-0.3.9 twitter.common.confluence-0.3.9
twitter.common.dirutil-0.3.9 twitter.common.lang-0.3.9 twitter.common.log-0.3.9
twitter.common.options-0.3.9
You are using pip version 8.1.2, however version 9.0.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
23:45:47 00:00 [main]
(To run a reporting server: ./pants server)
23:45:47 00:00 [setup]
23:45:47 00:00 [parse][32m
Executing tasks in goals: compile[0m
23:45:47 00:00 [compile]
23:45:47 00:00 [compile-prep-command]
23:45:47 00:00 [compile]
23:45:47 00:00 [python-eval]
23:45:47 00:00 [pythonstyle]
23:45:47 00:00 [cache] [32m
No cached artifacts for 42 targets.[0m[32m
Invalidated 42 targets.[0m
E261:ERROR PythonFile(src/main/python/apache/thermos/core/process.py):289 at
least two spaces before inline comment
| self._pid = self._platform.fork() # calls setup_child_subreaping
which can
T100:ERROR src/main/python/apache/thermos/common/process_util.py:069
Indentation of 4 instead of 2
| raise ValueError("libc not found")
FAILURE: 2 Python Style issues found. For import order related issues, please
try `./pants fmt.isort <targets>`
23:46:05 00:18 [complete][31m
FAILURE[0m
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On Nov. 3, 2016, 11:29 p.m., Zameer Manji wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53418/
> -----------------------------------------------------------
>
> (Updated Nov. 3, 2016, 11:29 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Stephan Erb.
>
>
> Bugs: AURORA-1808
> https://issues.apache.org/jira/browse/AURORA-1808
>
>
> Repository: aurora
>
>
> Description
> -------
>
> # Problem
>
> Processes can deamonize and escape the supervision of a coordinator. Using
> the Docker Containerizer or the Mesos Containerizer with pid isolation means
> that the processes will be come reparented to the sh process that launches
> the executor. For example:
>
> ```
> root@aurora:/# ps xf
> PID TTY STAT TIME COMMAND
> 48 ? Ss 0:00 /bin/bash
> 86 ? R+ 0:00 _ ps xf
> 1 ? Ss 0:00 /bin/sh -c ${MESOS_SANDBOX=.}/thermos_executor.pex
> --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config
> /home/vagrant/aurora/examples/va
> 5 ? Sl 0:02 python2.7 /mnt/mesos/sandbox/thermos_executor.pex
> --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config
> /home/vagrant/aurora/examples/vag
> 23 ? S 0:00 _ /usr/local/bin/python2.7
> /mnt/mesos/sandbox/thermos_runner.pex
> --task_id=www-data-devel-hello_docker_engine-0-bde5cdc7-8685-46fd-9078-4a86bd5be152
> --
> 29 ? Ss 0:00 _ /usr/local/bin/python2.7
> /mnt/mesos/sandbox/thermos_runner.pex
> --task_id=www-data-devel-hello_docker_engine-0-bde5cdc7-8685-46fd-9078-4a86bd5be15
> 32 ? S 0:00 | _ /bin/bash -c while true; do
> echo hello world sleep 10 done
> 81 ? S 0:00 | _ sleep 10
> 31 ? Ss 0:00 _ /usr/local/bin/python2.7
> /mnt/mesos/sandbox/thermos_runner.pex
> --task_id=www-data-devel-hello_docker_engine-0-bde5cdc7-8685-46fd-9078-4a86bd5be15
> 33 ? S 0:00 _ /bin/bash -c while true; do
> echo hello world sleep 10 done
> 82 ? S 0:00 _ sleep 10
> 47 ? S 0:00 python ./daemon.py
> ```
>
> # Solution
>
> Ensure processes that escape the supervision of the coordinator reparent to
> the runner who can send signals to them on task tear down. We do this by
> using the `PR_SET_CHILD_SUBREAPER` flag of `prctl(2)`.
>
> After this change the process tree looks like:
> ```
> root@aurora:/# ps xf
> PID TTY STAT TIME COMMAND
> 66 ? Ss 0:00 /bin/bash
> 70 ? R+ 0:00 _ ps xf
> 1 ? Ss 0:00 /bin/sh -c ${MESOS_SANDBOX=.}/thermos_executor.pex
> --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config
> /home/vagrant/aurora/examples/va
> 5 ? Sl 0:02 python2.7 /mnt/mesos/sandbox/thermos_executor.pex
> --announcer-ensemble localhost:2181 --announcer-zookeeper-auth-config
> /home/vagrant/aurora/examples/vag
> 23 ? S 0:00 _ /usr/local/bin/python2.7
> /mnt/mesos/sandbox/thermos_runner.pex
> --task_id=www-data-devel-hello_docker_engine-0-721406db-00f5-4c0c-915e-1dbc5568b849
> --
> 33 ? Ss 0:00 _ /usr/local/bin/python2.7
> /mnt/mesos/sandbox/thermos_runner.pex
> --task_id=www-data-devel-hello_docker_engine-0-721406db-00f5-4c0c-915e-1dbc5568b84
> 40 ? S 0:00 | _ /bin/bash -c while true; do
> echo hello world sleep 10 done
> 63 ? S 0:00 | _ sleep 10
> 36 ? Ss 0:00 _ /usr/local/bin/python2.7
> /mnt/mesos/sandbox/thermos_runner.pex
> --task_id=www-data-devel-hello_docker_engine-0-721406db-00f5-4c0c-915e-1dbc5568b84
> 37 ? S 0:00 | _ /bin/bash -c while true; do
> echo hello world sleep 10 done
> 62 ? S 0:00 | _ sleep 10
> 55 ? S 0:00 _ python ./daemon.py
>
> ```
>
> Now the runner is aware of the reparented procesess can can tear it down
> cleanly with a `SIGTERM`.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md d89ef2f641373ac229be693a21a2c0111e1f241a
> src/main/python/apache/thermos/common/process_util.py
> abd2c0ef35858d13971319b0a7436ce2293824ce
> src/main/python/apache/thermos/core/helper.py
> 68855e1e54ba1cd4456e18a36fb237ce6a468c34
> src/main/python/apache/thermos/core/process.py
> 3ec43e2719ef97026f399c4b2aa23002559b3153
> src/main/python/apache/thermos/core/runner.py
> 7b9013d11f6ff4172b6b7bf56e62299b0d11c977
> src/test/sh/org/apache/aurora/e2e/test_daemonizing_process.aurora
> PRE-CREATION
> src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 67702d2c0f2e18ee10dcb798b6d421050bd7d4ca
>
> Diff: https://reviews.apache.org/r/53418/diff/
>
>
> Testing
> -------
>
> src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Zameer Manji
>
>
