[GitHub] zookeeper pull request #687: Zookeeper 3169

TyqITstudent Sun, 04 Nov 2018 01:22:25 -0800

GitHub user TyqITstudent opened a pull request:

    https://github.com/apache/zookeeper/pull/687


    Zookeeper 3169

    **Problem:**
    1. Sometimes Zookeeper cluster will receive a lot of connections from 
clients, sometimes connection number even exceeds 1W. When zxid rolls over, the 
clients will reconnect and revalidate the session.
    2. In Zookeeper design structure, when follower server receives the session 
revalidation requests, it will send requests to leader server, which is 
designed to be responsible for session revalidation.
    3. In a short time, Leader will handle lots of requests. I use a tool to 
get the statistics, some clients need to wait over 20s. It is too long for some 
special clients, like ResourceManager.
    
    **Solution:**
    1. When zookeeper cluster finishes reelection(which will cost a few 
seconds). The leader will send the time point **TimeA** to followers. ï¼which 
is the approximate value of roll overï¼
    
    2.  Followers can judge the most session revalidations.  When the timeout 
of the session is less than **currentTime - timeA** , follower will put the 
session on the touchTable. ï¼Every half tickTime, followers will send sessions 
of touchTable to leader to validateï¼.
    
    3.  When the timeout of the session is larger than **currentTime - timeA**, 
the follower will send session revalidation request to leader right away.
    
    So the leader will receive fewer requests from followers.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/TyqITstudent/zookeeper ZOOKEEPER-3169

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zookeeper/pull/687.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #687
    
----
commit 620f9f4c22192e17d0fb9a4bb5486dcd2d8db2bd
Author: Flavio Paiva Junqueira <fpj@...>
Date:   2014-11-29T15:55:50Z

    ZOOKEEPER-2064 Prevent resource leak in various classes (Ted Yu via fpj)
    
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1642441 
13f79535-47bb-0310-9956-ffa450edef68

commit a9718779da86d9e61e2432e6e87c031db311f42f
Author: Michi Mutsuzaki <michim@...>
Date:   2015-02-11T07:01:42Z

    ZOOKEEPER-1949 recipes jar not included in the distribution package (Rakesh 
R via michim)
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1658890 
13f79535-47bb-0310-9956-ffa450edef68

commit 7b3ba5fca5802cec37605f8099c6fdead6d3c872
Author: Michi Mutsuzaki <michim@...>
Date:   2015-02-22T21:11:11Z

    ZOOKEEPER-2114 jute generated allocate_* functions are not externally 
visible (Tim Crowder via michim)
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1661562 
13f79535-47bb-0310-9956-ffa450edef68

commit 415bcd091228caef78e0d8b7bbdbbe67c48dee2f
Author: Michi Mutsuzaki <michim@...>
Date:   2015-02-22T22:08:16Z

    ZOOKEEPER-2073 Memory leak on zookeeper_close (Dave Gosselin via michim)
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1661573 
13f79535-47bb-0310-9956-ffa450edef68

commit 2c271faa1517ddaca0747cdf8760a1fb670f955f
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-04-07T00:50:58Z

    ZOOKEEPER-2056 Zookeeper 3.4.x and 3.5.0-alpha is not OSGi compliant 
(Deiwin Sarjas via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1671716 
13f79535-47bb-0310-9956-ffa450edef68

commit 87fc5e19bfae81508de96b4bb86d0aa1d5374c1c
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-04-18T00:47:56Z

    ZOOKEEPER-2146 BinaryInputArchive readString should check length before 
allocating memory (Hongchao Deng via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1674424 
13f79535-47bb-0310-9956-ffa450edef68

commit 7c3fe9d224e318a842b27113b7594cc7a33f88ce
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-05-03T18:02:51Z

    ZOOKEEPER-2174 JUnit4ZKTestRunner logs test failure for all exceptions
    
    JUnit4ZKTestRunner logs test failure for all exceptions, even if the test
    method is annotated with an expected exception (Chris Nauroth via rgs).
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1677463 
13f79535-47bb-0310-9956-ffa450edef68

commit c7830c532e6a8fe0d14c82f1ad1a3ebcdfc83187
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-05-15T04:06:54Z

    ZOOKEEPER-2126 Improve exit log messsage of EventThread and SendThread by 
adding SessionId (surendra singh lilhore via rakeshr)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1679491 
13f79535-47bb-0310-9956-ffa450edef68

commit 1169a5b5eca3982d24359c27f669af429ff71157
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-05-18T07:38:36Z

    ZOOKEEPER-1077: C client lib doesn't build on Solaris (Chris Nauroth via 
rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1679953 
13f79535-47bb-0310-9956-ffa450edef68

commit 6b8f0cd84e22494a664fcb7369d15286b083a1ed
Author: Michi Mutsuzaki <michim@...>
Date:   2015-05-24T06:33:43Z

    ZOOKEEPER-2186 QuorumCnxManager#receiveConnection may crash with random 
input (rgs via michim)
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1681417 
13f79535-47bb-0310-9956-ffa450edef68

commit ba96bd8a68c8eacab3a9e61b401cab0f35c7c314
Author: Michi Mutsuzaki <michim@...>
Date:   2015-05-24T06:37:32Z

    ZOOKEEPER-2124 Allow Zookeeper version string to have underscore '_' (Chris 
Nauroth via michim)
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1681418 
13f79535-47bb-0310-9956-ffa450edef68

commit 6ce171a99878f745c0b7f212e27c9064f95cc4e2
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-05-29T19:51:58Z

    ZOOKEEPER-2179: Typo in Watcher.java (Archana T via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1682541 
13f79535-47bb-0310-9956-ffa450edef68

commit e269cd5f4f76e8e2472de47354525674ad434659
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-06-02T20:43:29Z

    ZOOKEEPER-2096: C client builds with incorrect error codes in VisualStudio 
2010+
    (Vitaly Stakhovsky via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683180 
13f79535-47bb-0310-9956-ffa450edef68

commit a351209b3bd7e3123ce742789063784f6322d7fc
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-06-04T16:27:19Z

    ZOOKEEPER-2194: Let DataNode.getChildren() return an unmodifiable view of 
its
    children set (Hitoshi Mitake via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683589 
13f79535-47bb-0310-9956-ffa450edef68

commit 67cb77e74d0d8389a5d58781d7ed9b44ca3b3c38
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-06-05T18:44:25Z

    ZOOKEEPER-2205: Log type of unexpected quorum packet in learner handler loop
    (Hitoshi Mitake via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683838 
13f79535-47bb-0310-9956-ffa450edef68

commit 67019674379e6fac647d5eb23b694f1172d2849f
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-06-06T16:53:10Z

    ZOOKEEPER-2201: Network issues can cause cluster to hang due to 
near-deadlock
    (Donny Nadolny via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683931 
13f79535-47bb-0310-9956-ffa450edef68

commit 6fdb35ff1e1d3fbd1f5f22f9d4a8581ef8244851
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-06-11T18:14:21Z

    ZOOKEEPER-2213: Empty path in Set crashes server and prevents restart
    (Hongchao Deng via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1684958 
13f79535-47bb-0310-9956-ffa450edef68

commit 62888591c33968c3bf4fc79bb843cae9ff3ae5cf
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-06-13T00:49:48Z

    ZOOKEEPER-706: Large numbers of watches can cause session re-establishment 
to fail
    (Chris Thunes via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1685205 
13f79535-47bb-0310-9956-ffa450edef68

commit b407b1593ec6d1b3879f532a3aff700cec50bf75
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-06-18T18:58:48Z

    ZOOKEEPER-602: log all exceptions not caught by ZK threads
    (Rakesh R via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1686289 
13f79535-47bb-0310-9956-ffa450edef68

commit d66048c73a550aae8d34ccbae6ec384f3e8f74a3
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-07-06T15:48:24Z

    ZOOKEEPER-2224: Four letter command hangs when network is slow
    (Arshad Mohammad via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1689431 
13f79535-47bb-0310-9956-ffa450edef68

commit 0d254c8df43acae08c71edd299bda35b593d2f06
Author: Michi Mutsuzaki <michim@...>
Date:   2015-07-28T04:39:20Z

    ZOOKEEPER-2235: License update (fpj via michim)
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1692991 
13f79535-47bb-0310-9956-ffa450edef68

commit b2703be61c7d6e31ec2b535f5f130b1703eefad9
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-08-01T03:39:21Z

    ZOOKEEPER-2237 Port async multi to 3.4 branch (Ivan Kelly via rakeshr)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1693680 
13f79535-47bb-0310-9956-ffa450edef68

commit 8de5ad8d99a0f974a4393cc4eda546c66643dd96
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-08-05T20:17:18Z

    ZOOKEEPER-2239: JMX State from LocalPeerBean incorrect
    (Kevin Lee via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1694318 
13f79535-47bb-0310-9956-ffa450edef68

commit cfff3f06ae5c208ab566ab4fd941df835b880e53
Author: Hongchao Deng <hdeng@...>
Date:   2015-08-17T20:52:07Z

    ZOOKEEPER-1907 Improve Thread handling (Rakesh R via hdeng)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1696337 
13f79535-47bb-0310-9956-ffa450edef68

commit 908ea4084ea7b38b74b1fcaab7bfdf108d2a6093
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-08-25T05:13:25Z

    ZOOKEEPER-1927: zkServer.sh fails to read dataDir (and others)
    from zoo.cfg on Solaris 10 (grep issue, manifests as FAILED TO WRITE PID)
    (Chris Nauroth via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1697552 
13f79535-47bb-0310-9956-ffa450edef68

commit 3a7c6a5f010086a2305bbbde5efbf6ddb2e767ed
Author: Flavio Paiva Junqueira <fpj@...>
Date:   2015-09-03T23:00:20Z

    ZOOKEEPER-2033: zookeeper follower fails to start after a restart 
immediately following a new epoch (Asad Saeed via fpj)
    
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1701146 
13f79535-47bb-0310-9956-ffa450edef68

commit 1c1a2c785b4b18e639ff4497d96d721aaf11eb40
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-09-06T17:48:13Z

    ZOOKEEPER-2256: Zookeeper is not using specified JMX port in 
zkEnv.sh(Arshad Mohammad via rakeshr)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1701503 
13f79535-47bb-0310-9956-ffa450edef68

commit 8f326d5f5b2af8710ec64941fa5db7bca6bc06da
Author: RaÃºl GutiÃ©rrez SegalÃ©s <rgs@...>
Date:   2015-09-10T04:12:05Z

    ZOOKEEPER-2270: Allow MBeanRegistry to be overridden for better unit tests
    (Jordan Zimmerman via rgs)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1702157 
13f79535-47bb-0310-9956-ffa450edef68

commit 124670eb739d064e6bd01da451e92fe0084bdc6c
Author: Chris Nauroth <cnauroth@...>
Date:   2015-09-11T06:02:29Z

    ZOOKEEPER-2040: Server to log underlying cause of SASL connection problems. 
(Steve Loughran via cnauroth)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1702381 
13f79535-47bb-0310-9956-ffa450edef68

commit 9350096d66c4e405f312a453185471b2306fb9ee
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-09-17T07:12:00Z

    ZOOKEEPER-2245: SimpleSysTest test cases fails (Arshad Mohammad via rakeshr)
    
    git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1703504 
13f79535-47bb-0310-9956-ffa450edef68

----


---

[GitHub] zookeeper pull request #687: Zookeeper 3169

Reply via email to