-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58259/#review173923
-----------------------------------------------------------
Master (8a6e01c) is red with this patch.
./build-support/jenkins/build.sh
:commons:generateThriftResources
:commons:processResources
:commons:classes
:commons:jar
:compileJava/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java:74:
Note: Wrote forwarder
org.apache.aurora.scheduler.storage.log.WriteAheadStorageForwarder
@Forward({
^
Note: Writing
file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/org/apache/aurora/common/args/apt/cmdline.arg.info.txt.2
Note: Writing
file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/META-INF/compiler/resource-mappings/org.apache.aurora.common.args.apt.CmdLineProcessor
:generateBuildProperties
:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJavaNote:
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh
:jsHint
:checkstyleMain[ant:checkstyle] [ERROR]
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/updater/InstanceUpdater.java:41:
Wrong order for
'org.apache.aurora.scheduler.updater.StateEvaluator.Result.KILL_TASK_AND_EVALUATE_ON_STATE_CHANGE'
import. Order should be: java, javax, scala, com, net, org. Each group
should be separated by a single blank line. [ImportOrder]
FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at:
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output.
BUILD FAILED
Total time: 1 mins 38.378 secs
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On May 4, 2017, 4:21 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58259/
> -----------------------------------------------------------
>
> (Updated May 4, 2017, 4:21 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer
> Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> In the Dynamic Reservations review (and on the mailing list), I mentioned
> that we could implement update affinity with less complexity using the same
> technique as preemption. Here is how that would work.
>
> This just adds a simple wrapper around the preemptor's BiCache structure and
> then optimistically tries to keep an agent free for a task during the update
> process.
>
>
> Note: I don't bother even checking the resources before reserving the agent.
> I figure there is a chance the agent has enough room, and if not we'll catch
> it when we attempt to veto the offer. We need to always check the offer like
> this anyway in case constraints change. In the worst case it adds some delay
> in the rare cases you increase resources.
>
> We also don't persist the reservations, so if the Scheduler fails over during
> an update, the worst case is that any instances between the KILLED and
> ASSIGNED in-flight batch need to fall back to the current first-fit
> scheduling algorithm.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 1ee0d0167f0d212fa028f8ed0ddf20849703685c
> docs/operations/configuration.md f0581eade7296494d744b944ee8fddb94a5abaf6
> src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java
> f0b148cd158d61cd89cc51dca9f3fa4c6feb1b49
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java
> 211f1fc4ce46b853124f63d8ab8e37ac2f3d2d92
> src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java
> 203f62bacc47470545d095e4d25f7e0f25990ed9
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java
> a177b301203143539b052524d14043ec8a85a46d
> src/main/java/org/apache/aurora/scheduler/updater/InstanceAction.java
> b4cd01b3e03029157d5ca5d1d8e79f01296b57c2
>
> src/main/java/org/apache/aurora/scheduler/updater/InstanceActionHandler.java
> f25dc0c6d9c05833b9938b023669c9c36a489f68
> src/main/java/org/apache/aurora/scheduler/updater/InstanceUpdater.java
> c129896d8cd54abd2634e2a339c27921042b0162
>
> src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java
> e14112479807b4477b82554caf84fe733f62cf58
> src/main/java/org/apache/aurora/scheduler/updater/StateEvaluator.java
> c95943d242dc2f539778bdc9e071f342005e8de3
> src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java
> PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/updater/UpdaterModule.java
> 13cbdadad606d9acaadc541320b22b0ae538cc5e
> src/test/java/org/apache/aurora/scheduler/resources/ResourceBagTest.java
> c1638265373e8f4cb508c08d5a62e07027eff9c8
>
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java
> fa1a81785802b82542030e1aae786fe9570d9827
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java
> cf2d25ec2e407df7159e0021ddb44adf937e1777
> src/test/java/org/apache/aurora/scheduler/updater/AddTaskTest.java
> b2c4c66850dd8f35e06a631809530faa3b776252
> src/test/java/org/apache/aurora/scheduler/updater/InstanceUpdaterTest.java
> df1f8394b824dbb7b2745fcccdab5adaafdf6e6c
> src/test/java/org/apache/aurora/scheduler/updater/JobUpdaterIT.java
> 30b44f88a5b8477e917da21d92361aea1a39ceeb
> src/test/java/org/apache/aurora/scheduler/updater/KillTaskTest.java
> 833fd62c870f96b96343ee5e0eed0d439536381f
>
> src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java
> PRE-CREATION
>
> src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58259/diff/6/
>
>
> Testing
> -------
>
> ./gradlew build
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> File Attachments
> ----------------
>
> Cache utilization over time
>
> https://reviews.apache.org/media/uploaded/files/2017/04/25/7b41bd2b-4151-482c-9de2-9dee67c34133__declining-cache-hits.png
> Offer rate from Mesos over time
>
> https://reviews.apache.org/media/uploaded/files/2017/04/25/b107d964-ee7d-435a-a3d9-2b54f6eac3fa__consistent-offer-rate.png
> Async task workload (scaled) correlation with degraded cache utilization
>
> https://reviews.apache.org/media/uploaded/files/2017/04/25/7eaf37ac-fbf3-40eb-b3f6-90e914a3936f__async-task-correlation.png
> cache hit rate before and after scheduler tuning
>
> https://reviews.apache.org/media/uploaded/files/2017/05/02/39998e8d-2a75-4f5d-bfc0-bb93011407af__Screen_Shot_2017-05-01_at_6.30.18_PM.png
> JobUpdateControllerImpl bottleneck
>
> https://reviews.apache.org/media/uploaded/files/2017/05/02/f93484bd-c99e-4c01-9f8a-f0ad867adb26__Screen_Shot_2017-05-02_at_3.33.39_PM.png
>
>
> Thanks,
>
> David McLaughlin
>
>
