-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60376/#review179153
-----------------------------------------------------------
Master (a922b05) is red with this patch.
./build-support/jenkins/build.sh
Downloading six-1.10.0-py2.py3-none-any.whl
Collecting ansicolors==1.0.2 (from pantsbuild.pants==1.3.0.dev3)
Downloading ansicolors-1.0.2.tar.gz
Collecting packaging==16.7 (from pantsbuild.pants==1.3.0.dev3)
Downloading packaging-16.7-py2.py3-none-any.whl
Collecting pathspec==0.3.4 (from pantsbuild.pants==1.3.0.dev3)
Downloading pathspec-0.3.4.tar.gz
Collecting scandir==1.2 (from pantsbuild.pants==1.3.0.dev3)
Downloading scandir-1.2.zip
Collecting twitter.common.dirutil<0.4,>=0.3.1 (from
pantsbuild.pants==1.3.0.dev3)
Downloading twitter.common.dirutil-0.3.9.tar.gz
Collecting psutil==4.3.0 (from pantsbuild.pants==1.3.0.dev3)
Downloading psutil-4.3.0.tar.gz (316kB)
Collecting requests<2.6,>=2.5.0 (from pantsbuild.pants==1.3.0.dev3)
Downloading requests-2.5.3-py2.py3-none-any.whl (468kB)
Collecting pystache==0.5.3 (from pantsbuild.pants==1.3.0.dev3)
Downloading pystache-0.5.3.tar.gz (74kB)
Collecting pex==1.1.16 (from pantsbuild.pants==1.3.0.dev3)
Downloading pex-1.1.16-py2.py3-none-any.whl (105kB)
Collecting docutils<0.13,>=0.12 (from pantsbuild.pants==1.3.0.dev3)
Downloading docutils-0.12.tar.gz (1.6MB)
Collecting Markdown==2.1.1 (from pantsbuild.pants==1.3.0.dev3)
Downloading Markdown-2.1.1.tar.gz (242kB)
Collecting Pygments==1.4 (from pantsbuild.pants==1.3.0.dev3)
Downloading Pygments-1.4.tar.gz (3.5MB)
Collecting twitter.common.confluence<0.4,>=0.3.1 (from
pantsbuild.pants==1.3.0.dev3)
Downloading twitter.common.confluence-0.3.9.tar.gz
Collecting fasteners==0.14.1 (from pantsbuild.pants==1.3.0.dev3)
Downloading fasteners-0.14.1-py2.py3-none-any.whl
Collecting coverage<3.8,>=3.7 (from pantsbuild.pants==1.3.0.dev3)
Downloading coverage-3.7.1.tar.gz (284kB)
Complete output from command python setup.py egg_info:
running egg_info
error: error in 'egg_base' option: 'pip-egg-info' does not exist or is not
a directory
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in
/tmp/pip-build-asSx_T/coverage/
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/../../pants:
line 99:
/home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.3.0.dev3/bin/python:
No such file or directory
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On June 28, 2017, 7:01 p.m., Reza Motamedi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60376/
> -----------------------------------------------------------
>
> (Updated June 28, 2017, 7:01 p.m.)
>
>
> Review request for Aurora, David McLaughlin, Joshua Cohen, Jordan Ly, and
> Santhosh Kumar Shanmugham.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> # Observer task page to load consumption info from history
>
> Resource consumptions of Thermos Processes are periodically calculated by
> TaskResourceMonitor threads (one thread per Thermos task). This information
> is used to display a (semi) fresh state of the tasks running on a host in the
> Observer host page, aka landing page. An aggregate history of the
> consumptions is kept at the task level, although TaskResourceMonitor needs to
> first collect the resource at the Process level and then aggregate them.
>
> On the other hand, when an Observer _task page_ is visited, the resources
> consumption of Thermos Processes within that task are calculated again and
> displayed without being aggregated. This can become very slow since time to
> complete resource calculation is affected by the load on the host.
>
> By applying this patch we take advantage of the periodic work and fulfill
> information resource requested in Observer task page from already collected
> resource consumptions.
>
>
> Diffs
> -----
>
> src/main/python/apache/thermos/monitoring/resource.py
> 434666696e600a0e6c19edd986c86575539976f2
> src/test/python/apache/aurora/executor/common/test_resource_manager.py
> a898e4d81d34d1e30e39db1be1a66bc9e0ab1a35
> src/test/python/apache/thermos/monitoring/test_resource.py
> d794a998f1d9fc52ba260cd31ac444aee7f8ed28
>
>
> Diff: https://reviews.apache.org/r/60376/diff/2/
>
>
> Testing
> -------
>
> I stress tested this patch on a host that had a slow Observer page.
> Interestingly, I did not need to do much to make the Observer slow. There are
> a few points to be made clear first.
> - We at Twitter limit the resources allocated to the Observer using
> `systemd`. The observer is allowed to use only 20% of a CPU core. The
> attached screen shots are from such a setup.
> - Having assigned 20% of a cpu core to Observer, starting only 8 `task`s,
> each with 3 `process`es is enough to make the Observer slow; 11secs to load
> `task page`.
>
>
> File Attachments
> ----------------
>
> without the patch -- Screen Shot 2017-06-22 at 1.11.12 PM.png
>
> https://reviews.apache.org/media/uploaded/files/2017/06/22/03968028-a2f5-4a99-ba57-b7a41c471436__without_the_patch_--_Screen_Shot_2017-06-22_at_1.11.12_PM.png
> with the patch -- Screen Shot 2017-06-22 at 1.07.41 PM.png
>
> https://reviews.apache.org/media/uploaded/files/2017/06/22/5962c018-27d3-4463-a277-f6ad48b7f2d7__with_the_patch_--_Screen_Shot_2017-06-22_at_1.07.41_PM.png
>
>
> Thanks,
>
> Reza Motamedi
>
>
