-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59699/#review176695
-----------------------------------------------------------
Master (e76862a) is red with this patch.
./build-support/jenkins/build.sh
:commons:generateThriftResources
:commons:processResources
:commons:classes
:commons:jar
:compileJava/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java:74:
Note: Wrote forwarder
org.apache.aurora.scheduler.storage.log.WriteAheadStorageForwarder
@Forward({
^
Note: Writing
file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/org/apache/aurora/common/args/apt/cmdline.arg.info.txt.2
Note: Writing
file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/META-INF/compiler/resource-mappings/org.apache.aurora.common.args.apt.CmdLineProcessor
:generateBuildProperties
:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJavaNote:
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh
:jsHint
:checkstyleMain[ant:checkstyle] [ERROR]
/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java:21:8:
Unused import - java.util.stream.Collectors. [UnusedImports]
FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at:
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output.
BUILD FAILED
Total time: 1 mins 27.917 secs
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On June 2, 2017, 1:41 a.m., Kai Huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59699/
> -----------------------------------------------------------
>
> (Updated June 2, 2017, 1:41 a.m.)
>
>
> Review request for Aurora, David McLaughlin and Santhosh Kumar.
>
>
> Bugs: AURORA-1929
> https://issues.apache.org/jira/browse/AURORA-1929
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Improve task history pruning by batch deleting tasks.
>
> The `'aurora_admin prune_tasks'` endpoint seems to be very slow when the
> cluster has a large number of inactive tasks.
>
> This CR batches all removeTasks operations and execute them all at once to
> avoid additional cost of coalescing. The fix will also benefit implicit task
> history pruning since it has similar underlying implementation. See
> https://issues.apache.org/jira/browse/AURORA-1929 for more information and
> details.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java
> 73878210f9028901fda3b08e66c6a63c24260d35
>
>
> Diff: https://reviews.apache.org/r/59699/diff/3/
>
>
> Testing
> -------
>
> __unit_tests:__
>
> ./build-support/jenkins/build.sh
>
> No unit tests were created for this patch since it does not add new
> functionalities or alter the interface, but improves the efficiency of the
> existing code.
>
> __e2e tests:__
>
> Attached was a screenshot of the task history pruning benchmark obtained from
> a scale test in Twitter's test cluster.
>
> - Before applying this patch, the task history pruning takes ~30 minutes on
> 130K tasks.
>
> - After applying the patch, the pruning takes ~1 minute.
>
>
> File Attachments
> ----------------
>
> task_history_pruning_benchmark.png
>
> https://reviews.apache.org/media/uploaded/files/2017/06/01/74eb5104-d338-4530-abd2-b82fbdc6bf84__task_history_pruning_benchmark.png
>
>
> Thanks,
>
> Kai Huang
>
>
