----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60376/#review180795 -----------------------------------------------------------
Master (243d6fa) is red with this patch. ./build-support/jenkins/build.sh /home/jenkins/jenkins-slave/workspace/AuroraBot + exec /usr/bin/python2.7 /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/virtualenv-15.0.2/virtualenv.py --no-download /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/isort.venv New python executable in /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/isort.venv/bin/python2.7 Also creating executable in /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/isort.venv/bin/python Installing setuptools, pip, wheel...done. Collecting isort==4.0.0 /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/isort.venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. SNIMissingWarning /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/isort.venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Downloading isort-4.0.0-py2.py3-none-any.whl Installing collected packages: isort Successfully installed isort-4.0.0 /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/isort.venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning You are using pip version 8.1.2, however version 9.0.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. ERROR: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/monitoring/test_resource.py Imports are incorrectly sorted. --- /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/monitoring/test_resource.py:before 2017-07-18 06:31:11.809995 +++ /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/monitoring/test_resource.py:after 2017-07-18 06:36:40.261237 @@ -14,8 +14,10 @@ from time import time from unittest import TestCase + import mock import pytest +from twitter.common.quantity import Amount, Time from apache.thermos.monitoring.monitor import TaskMonitor from apache.thermos.monitoring.process import ProcessSample @@ -27,8 +29,6 @@ ) from gen.apache.thermos.ttypes import ProcessStatus - -from twitter.common.quantity import Amount, Time class TestResourceHistoryProvider(TestCase): I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On July 18, 2017, 2:26 p.m., Reza Motamedi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60376/ > ----------------------------------------------------------- > > (Updated July 18, 2017, 2:26 p.m.) > > > Review request for Aurora, David McLaughlin, Joshua Cohen, Jordan Ly, and > Santhosh Kumar Shanmugham. > > > Repository: aurora > > > Description > ------- > > # Observer task page to load consumption info from history > > Resource consumptions of Thermos Processes are periodically calculated by > TaskResourceMonitor threads (one thread per Thermos task). This information > is used to display a (semi) fresh state of the tasks running on a host in the > Observer host page, aka landing page. An aggregate history of the > consumptions is kept at the task level, although TaskResourceMonitor needs to > first collect the resource at the Process level and then aggregate them. > > On the other hand, when an Observer _task page_ is visited, the resources > consumption of Thermos Processes within that task are calculated again and > displayed without being aggregated. This can become very slow since time to > complete resource calculation is affected by the load on the host. > > By applying this patch we take advantage of the periodic work and fulfill > information resource requested in Observer task page from already collected > resource consumptions. > > > Diffs > ----- > > src/main/python/apache/thermos/monitoring/resource.py > 434666696e600a0e6c19edd986c86575539976f2 > src/test/python/apache/aurora/executor/common/test_resource_manager.py > a898e4d81d34d1e30e39db1be1a66bc9e0ab1a35 > src/test/python/apache/thermos/monitoring/test_resource.py > d794a998f1d9fc52ba260cd31ac444aee7f8ed28 > > > Diff: https://reviews.apache.org/r/60376/diff/4/ > > > Testing > ------- > > I stress tested this patch on a host that had a slow Observer page. > Interestingly, I did not need to do much to make the Observer slow. There are > a few points to be made clear first. > - We at Twitter limit the resources allocated to the Observer using > `systemd`. The observer is allowed to use only 20% of a CPU core. The > attached screen shots are from such a setup. > - Having assigned 20% of a cpu core to Observer, starting only 8 `task`s, > each with 3 `process`es is enough to make the Observer slow; 11secs to load > `task page`. > > > File Attachments > ---------------- > > without the patch -- Screen Shot 2017-06-22 at 1.11.12 PM.png > > https://reviews.apache.org/media/uploaded/files/2017/06/22/03968028-a2f5-4a99-ba57-b7a41c471436__without_the_patch_--_Screen_Shot_2017-06-22_at_1.11.12_PM.png > with the patch -- Screen Shot 2017-06-22 at 1.07.41 PM.png > > https://reviews.apache.org/media/uploaded/files/2017/06/22/5962c018-27d3-4463-a277-f6ad48b7f2d7__with_the_patch_--_Screen_Shot_2017-06-22_at_1.07.41_PM.png > > > Thanks, > > Reza Motamedi > >
