[
https://issues.apache.org/jira/browse/ZOOKEEPER-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116336#comment-15116336
]
Hadoop QA commented on ZOOKEEPER-2024:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12784274/ZOOKEEPER-2024.patch
against trunk revision 1726354.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 11 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3014//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3014//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3014//console
This message is automatically generated.
> Major throughput improvement with mixed workloads
> -------------------------------------------------
>
> Key: ZOOKEEPER-2024
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2024
> Project: ZooKeeper
> Issue Type: Improvement
> Components: quorum, server
> Reporter: Kfir Lev-Ari
> Assignee: Kfir Lev-Ari
> Attachments: ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch,
> ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch,
> ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch
>
>
> The patch is applied to the commit processor, and solves two problems:
> 1. Stalling - once the commit processor encounters a local write request, it
> stalls local processing of all sessions until it receives a commit of that
> request from the leader.
> In mixed workloads, this severely hampers performance as it does not allow
> read-only sessions to proceed at faster speed than read-write ones.
> 2. Starvation - as long as there are read requests to process, older remote
> committed write requests are starved.
> This occurs due to a bug fix
> (https://issues.apache.org/jira/browse/ZOOKEEPER-1505) that forces processing
> of local read requests before handling any committed write. The problem is
> only manifested under high local read load.
> Our solution solves these two problems. It improves throughput in mixed
> workloads (in our tests, by up to 8x), and reduces latency, especially higher
> percentiles (i.e., slowest requests).
> The main idea is to separate sessions that inherently need to stall in order
> to enforce order semantics, from ones that do not need to stall. To this end,
> we add data structures for buffering and managing pending requests of stalled
> sessions; these requests are moved out of the critical path to these data
> structures, allowing continued processing of unaffected sessions.
> In order to avoid starvation, our solution prioritizes committed write
> requests over reads, and enforces fairness among read requests of sessions.
> Please see the docs:
> 1)
> https://docs.google.com/document/d/1oXJiSt9VqL35hCYQRmFuC63ETd0F_g6uApzocgkFe3Y/edit?usp=sharing
> - includes a detailed description of the new commit processor algorithm.
> 2) The attached patch implements our solution, and a collection of related
> unit tests (https://reviews.apache.org/r/25160)
> 3)
> https://docs.google.com/spreadsheets/d/11mmobkIf-0czIyEEwgytwqRme5OH8tmZcb4EBcsMZ_w/edit?usp=sharing
> - new performance results.
> https://docs.google.com/spreadsheets/d/1vmdfsq4WLr92BQO-CGcualE0KhAtjIu3bCaVwYajLo8/edit?usp=sharing
> - shows (old) performance results of running system tests on the patched ZK
> using the patched system test from
> https://issues.apache.org/jira/browse/ZOOKEEPER-2023.
> See also https://issues.apache.org/jira/browse/ZOOKEEPER-1609
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
