[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15141685#comment-15141685
 ] 

Hadoop QA commented on ZOOKEEPER-2024:
--------------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12787326/ZOOKEEPER-2024.patch
  against trunk revision 1729259.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 11 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 2.0.3) 
warnings.

    +1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3043//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3043//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3043//console

This message is automatically generated.

> Major throughput improvement with mixed workloads
> -------------------------------------------------
>
>                 Key: ZOOKEEPER-2024
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2024
>             Project: ZooKeeper
>          Issue Type: Improvement
>          Components: quorum, server
>            Reporter: Kfir Lev-Ari
>            Assignee: Kfir Lev-Ari
>         Attachments: ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch, 
> ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch, 
> ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch, ZOOKEEPER-2024.patch, 
> ZOOKEEPER-2024.patch
>
>
> The patch is applied to the commit processor, and solves two problems:
> 1. Stalling - once the commit processor encounters a local write request, it 
> stalls local processing of all sessions until it receives a commit of that 
> request from the leader. 
> In mixed workloads, this severely hampers performance as it does not allow 
> read-only sessions to proceed at faster speed than read-write ones.
> 2. Starvation - as long as there are read requests to process, older remote 
> committed write requests are starved. 
> This occurs due to a bug fix 
> (https://issues.apache.org/jira/browse/ZOOKEEPER-1505) that forces processing 
> of local read requests before handling any committed write. The problem is 
> only manifested under high local read load. 
> Our solution solves these two problems. It improves throughput in mixed 
> workloads (in our tests, by up to 8x), and reduces latency, especially higher 
> percentiles (i.e., slowest requests). 
> The main idea is to separate sessions that inherently need to stall in order 
> to enforce order semantics, from ones that do not need to stall. To this end, 
> we add data structures for buffering and managing pending requests of stalled 
> sessions; these requests are moved out of the critical path to these data 
> structures, allowing continued processing of unaffected sessions. 
> In order to avoid starvation, our solution prioritizes committed write 
> requests over reads, and enforces fairness among read requests of sessions. 
> Please see the docs:  
> 1) 
> https://docs.google.com/document/d/1oXJiSt9VqL35hCYQRmFuC63ETd0F_g6uApzocgkFe3Y/edit?usp=sharing
>  - includes a detailed description of the new commit processor algorithm.
> 2) The attached patch implements our solution, and a collection of related 
> unit tests (https://reviews.apache.org/r/25160)
> 3) 
> https://docs.google.com/spreadsheets/d/11mmobkIf-0czIyEEwgytwqRme5OH8tmZcb4EBcsMZ_w/edit?usp=sharing
>  - new performance results.
> https://docs.google.com/spreadsheets/d/1vmdfsq4WLr92BQO-CGcualE0KhAtjIu3bCaVwYajLo8/edit?usp=sharing
>  - shows (old) performance results of running system tests on the patched ZK 
> using the patched system test from 
> https://issues.apache.org/jira/browse/ZOOKEEPER-2023. 
> See also https://issues.apache.org/jira/browse/ZOOKEEPER-1609



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to