[
https://issues.apache.org/jira/browse/ZOOKEEPER-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362102#comment-14362102
]
Camille Fournier commented on ZOOKEEPER-1865:
---------------------------------------------
So, the test as written does not actually exhibit the error we're fixing; if we
revert the meaningful change you've proposed to Learner it will still pass. I
updated it a bit to get it to fail with the (mostly) old code (modulo some
helper methods you wrote), and pass with the new code. Have attached. [~michim]
if you have a chance to look at this quickly it would be nice to get this into
3.5.1
> Fix retry logic in Learner.connectToLeader()
> ---------------------------------------------
>
> Key: ZOOKEEPER-1865
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1865
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Reporter: Thawan Kooburat
> Assignee: Edward Carter
> Fix For: 3.5.1
>
> Attachments: ZOOKEEPER-1865-nanoTime.patch, ZOOKEEPER-1865.patch
>
>
> We discovered a long leader election time today in one of our prod ensemble.
> Here is the description of the event.
> Before the old leader goes down, it is able to announce notification message.
> So 3 out 5 (including the old leader) elected the old leader to be a new
> leader for the next epoch. While, the old leader is being rebooted, 2 other
> machines are trying to connect to the old leader. So the quorum couldn't
> form until those 2 machines give up and move to the next round of leader
> election.
> This is because Learner.connectToLeader() use a simple retry logic. The
> contract for this method is that it should never spend longer that initLimit
> trying to connect to the leader. In our outage, each sock.connect() is
> probably blocked for initLimit and it is called 5 times.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
