-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63536/#review191608
-----------------------------------------------------------
Master (46b1112) is red with this patch.
./build-support/jenkins/build.sh
Running setup.py bdist_wheel for twitter.common.log: finished with status
'done'
Stored in directory:
/home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/80/e3/3a/297d8950fcbd822ab5a6a62175818fab38b668cc5a86dbd0b0
Running setup.py bdist_wheel for pycparser: started
Running setup.py bdist_wheel for pycparser: finished with status 'done'
Stored in directory:
/home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/95/14/9a/5e7b9024459d2a6600aaa64e0ba485325aff7a9ac7489db1b6
Running setup.py bdist_wheel for twitter.common.options: started
Running setup.py bdist_wheel for twitter.common.options: finished with status
'done'
Stored in directory:
/home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/17/41/80/c4811d8c1c7ca7007e520c3399872fc340f45c3a26a6a23e6a
Successfully built pantsbuild.pants twitter.common.collections setproctitle
ansicolors pathspec scandir twitter.common.dirutil psutil pystache docutils
Markdown Pygments twitter.common.confluence coverage lmdb pywatchman
twitter.common.lang twitter.common.log pycparser twitter.common.options
Installing collected packages: twitter.common.lang, twitter.common.collections,
setproctitle, setuptools, six, ansicolors, pyparsing, packaging, pathspec,
scandir, twitter.common.dirutil, psutil, requests, pystache, pex, docutils,
Markdown, Pygments, twitter.common.options, twitter.common.log,
twitter.common.confluence, monotonic, fasteners, coverage, pycparser, cffi,
lmdb, pywatchman, futures, pantsbuild.pants
Found existing installation: setuptools 21.2.1
Uninstalling setuptools-21.2.1:
Successfully uninstalled setuptools-21.2.1
Successfully installed Markdown-2.1.1 Pygments-1.4 ansicolors-1.0.2 cffi-1.7.0
coverage-3.7.1 docutils-0.12 fasteners-0.14.1 futures-3.0.5 lmdb-0.89
monotonic-1.4 packaging-16.7 pantsbuild.pants-1.3.0.dev3 pathspec-0.3.4
pex-1.1.16 psutil-4.3.0 pycparser-2.18 pyparsing-2.2.0 pystache-0.5.3
pywatchman-1.3.0 requests-2.5.3 scandir-1.2 setproctitle-1.1.10
setuptools-30.0.0 six-1.11.0 twitter.common.collections-0.3.9
twitter.common.confluence-0.3.9 twitter.common.dirutil-0.3.9
twitter.common.lang-0.3.9 twitter.common.log-0.3.9 twitter.common.options-0.3.9
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
18:00:44 00:00 [main]
(To run a reporting server: ./pants server)
18:00:44 00:00 [setup]
18:00:44 00:00 [parse][32m
Executing tasks in goals: compile[0m
18:00:44 00:00 [compile]
18:00:44 00:00 [compile-prep-command]
18:00:44 00:00 [prep_command]
18:00:48 00:04 [compile]
18:00:48 00:04 [python-eval]
18:00:48 00:04 [pythonstyle]
18:00:49 00:05 [cache] [32m
No cached artifacts for 42 targets.[0m[32m
Invalidated 42 targets.[0m
F401:ERROR src/main/python/apache/aurora/client/cli/jobs.py:066
'PystachioPartitionPolicy' imported but unused
|from apache.aurora.config.schema.base import PartitionPolicy as
PystachioPartitionPolicy
FAILURE: 1 Python Style issues found. For import order related issues, please
try `./pants fmt.isort <targets>`
18:01:05 00:21 [complete][31m
FAILURE[0m
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On Nov. 21, 2017, 5:50 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63536/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2017, 5:50 p.m.)
>
>
> Review request for Aurora, Jordan Ly, Santhosh Kumar Shanmugham, and Bill
> Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is my prototype code for adding partition-awareness to Aurora. There is
> a proposal document to accompany this here:
> https://docs.google.com/document/d/1E3GlsVTJLEMAkDWk2_PTxzkRZcapb8nF_5q5AADQI7g/edit#
>
> I'd like feedback on the high-level approach before adding unit tests,
> metrics, logging, etc.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md d653b797533735fc936a1cff2e005813757e1290
> api/src/main/thrift/org/apache/aurora/gen/api.thrift
> 1d369263d779b549b9304018437f535ddc855966
> examples/vagrant/upstart/aurora-scheduler.conf
> 5ca3caef03b6632cd4dbf47711b1ef183f6a6449
> src/main/java/org/apache/aurora/scheduler/base/Conversions.java
> 33cc012a2cad929b0dd1ce236597b870cfc5aba0
> src/main/java/org/apache/aurora/scheduler/base/Jobs.java
> 8d5f4e57c6b4f847cb74471f246fd0b7dd0cbc36
> src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java
> 7c223eae69503fe1d5bf34c430438637abcbcb9b
>
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java
> 5e83b2acdde7198d16427d4031e9772f78612554
> src/main/java/org/apache/aurora/scheduler/state/PartitionManager.java
> PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/SideEffect.java
> b91a0852e968b4aa9d74801601671cb61af3648a
> src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java
> 9989ed441cd9bc442e6472768880ce7924c3bdd9
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java
> c03fff11ea3a4086f9daaa8b07315006c1b481e4
> src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java
> eb4fe7d78ad1e6ec430c428df527bd0cf3a053c1
> src/main/python/apache/aurora/client/cli/jobs.py
> b79ae56bee0e5692cacf1e66f4a4126b06aaffdc
> src/main/python/apache/aurora/config/schema/base.py
> 18ce826363009e1cc0beac5cce4edf42610d487e
> src/main/python/apache/aurora/config/thrift.py
> bedf8cd6641e1b1a930602791b758d584af4891c
> src/test/java/org/apache/aurora/scheduler/base/JobsTest.java
> 13f656f241a8a9a3d339f4053f165070c2669ef3
> src/test/java/org/apache/aurora/scheduler/config/CommandLineTest.java
> c2d875bb5c393dd95d75251fe86dc649ceba7bd9
>
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java
> 7b0429109e9a7795e559db264e7737fc55ff0169
> src/test/java/org/apache/aurora/scheduler/state/PartitionManagerTest.java
> PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java
> 0366cd6e9ddba0c3b9c88ffb50738767a352a17a
> src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java
> 8d6c3fff0af2df39bb929f760b862a2edf5d6fca
> src/test/python/apache/aurora/client/cli/test_task.py
> 186cb2737ba8e169819b7d54f86a7344a669b6cb
> src/test/python/apache/aurora/config/test_thrift.py
> 7a1567a9b67917072bb0ba3eea5857e968375f4d
> src/test/sh/org/apache/aurora/e2e/partition_aware.aurora PRE-CREATION
> src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> f0819fb7ac758ad1229a76fd9794b393400e9f63
>
>
> Diff: https://reviews.apache.org/r/63536/diff/8/
>
>
> Testing
> -------
>
> Manual testing in Vagrant by stopping and starting the Mesos agent. With
> three jobs:
>
> 1) No PartitionPolicy (verified existing behavior of moving from PARTITIONED
> directly to LOST)
> 2) PartitionPolicy with custom delay_secs (verified sat in PARTITIONED for a
> while before moving to LOST)
> 3) PartitionPolicy with reschedule=False (verified sat in PARTITIONED
> indefinitely)
>
> I also verified tasks are able to transition to various states (back to
> RUNNING or moving to FAILED, etc.) when you turn the agent back on.
>
>
> File Attachments
> ----------------
>
> Task in PARTITIONED state
>
> https://reviews.apache.org/media/uploaded/files/2017/11/07/02c7fc72-b11d-4ef9-a86b-914e748cad99__Screen_Shot_2017-11-07_at_11.23.41_AM.png
> Task back into running when partition resolved
>
> https://reviews.apache.org/media/uploaded/files/2017/11/07/a0413f54-1572-4410-a386-0a22e78fab13__Screen_Shot_2017-11-07_at_11.26.02_AM.png
> Compaction of PARTITIONED cycles (note timestamps)
>
> https://reviews.apache.org/media/uploaded/files/2017/11/07/edec32e5-b3ec-4fdc-b93f-5449519805ae__Screen_Shot_2017-11-07_at_11.27.47_AM.png
>
>
> Thanks,
>
> David McLaughlin
>
>
