[
https://issues.apache.org/jira/browse/ZOOKEEPER-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13653142#comment-13653142
]
Hadoop QA commented on ZOOKEEPER-1697:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12582498/ZOOKEEPER-1697.patch
against trunk revision 1463329.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1474//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1474//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1474//console
This message is automatically generated.
> large snapshots can cause continuous quorum failure
> ---------------------------------------------------
>
> Key: ZOOKEEPER-1697
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1697
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.3, 3.5.0
> Reporter: Patrick Hunt
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.5.0, 3.4.6
>
> Attachments: ZOOKEEPER-1697_branch34.patch,
> ZOOKEEPER-1697_branch34.patch, ZOOKEEPER-1697.patch, ZOOKEEPER-1697.patch
>
>
> I keep seeing this on the leader:
> 2013-04-30 01:18:39,754 INFO
> org.apache.zookeeper.server.quorum.Leader: Shutdown called
> java.lang.Exception: shutdown Leader! reason: Only 0 followers, need 2
> at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:447)
> at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:422)
> at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:753)
> The followers are downloading the snapshot when this happens, and are
> trying to do their first ACK to the leader, the ack fails with broken
> pipe.
> In this case the snapshots are large and the config has increased the
> initLimit. syncLimit is small - 10 or so with ticktime of 2000. Note
> this is 3.4.3 with ZOOKEEPER-1521 applied.
> I originally speculated that
> https://issues.apache.org/jira/browse/ZOOKEEPER-1521 might be related.
> I thought I might have broken something for this environment. That
> doesn't look to be the case.
> As it looks now it seems that 1521 didn't go far enough. The leader
> verifies that all followers have ACK'd to the leader within the last
> "syncLimit" time period. This runs all the time in the background on
> the leader to identify the case where a follower drops. In this case
> the followers take so long to load the snapshot that this check fails
> the very first time, as a result the leader drops (not enough ack'd
> followers w/in the sync limit) and re-election happens. This repeats
> forever. (the above error)
> this is the call:
> org.apache.zookeeper.server.quorum.LearnerHandler.synced() that's at
> odds.
> look at setting of tickOfLastAck in
> org.apache.zookeeper.server.quorum.LearnerHandler.run()
> It's not set until the follower first acks - in this case I can see
> that the followers are not getting to the ack prior to the leader
> shutting down due to the error log above.
> It seems that sync() should probably use the init limit until the
> first ack comes in from the follower. I also see that while tickOfLastAck and
> leader.self.tick is shared btw two threads there is no synchronization of the
> shared resources.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
