[
https://issues.apache.org/jira/browse/ZOOKEEPER-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024815#comment-17024815
]
Suhas Dantkale edited comment on ZOOKEEPER-3707 at 1/28/20 3:44 AM:
--------------------------------------------------------------------
In step(5) above, SID 3 actually must be waiting to see if there are any
changes to the proposed leader.
{color:#0747a6}_// Verify if there is any change in the proposed leader_{color}
{color:#0747a6} _while((n = recvqueue.poll(finalizeWait,_{color}
{color:#0747a6} _TimeUnit.MILLISECONDS)) != null){_{color}
{color:#0747a6} _if(totalOrderPredicate(n.leader, n.zxid, n.peerEpoch,_{color}
{color:#0747a6} _proposedLeader, proposedZxid, proposedEpoch))_{color}
{color:#0747a6}_{ recvqueue.put(n); break; }_{color}
{color:#0747a6}_}_{color}
So my question is :
Should SID 3 come out of FLE (as LEADING) without waiting long enough to let
SID 1 and SID 2 to change their vote to SID 4 (?)
finalizeWait is 200 Ms.
Had SID 3 waited long enough, it SID 3 would have not selected itself as
prospective leader. Instead would have learned that SID 1 and 2 have changed
their votes to make SID 4 as the leader. And meantime would have also obtained
the Notification from SID 4.
So if I increase _finalizeWait_ as a dirty hack/workaround _,_ would it solve
this problem_?_
OR
Somehow, SID 4 and SID 1 should have learned that 1 and 3 are out of election
and instead of verifying that a majority are following SID 3, should have
somehow learn to break a tie(?)
was (Author: suhas.dantkale):
In step(5) above, SID 3 actually must be waiting to see if there are any
changes to the proposed leader.
{color:#0747a6}_// Verify if there is any change in the proposed leader_{color}
{color:#0747a6} _while((n = recvqueue.poll(finalizeWait,_{color}
{color:#0747a6} _TimeUnit.MILLISECONDS)) != null){_{color}
{color:#0747a6} _if(totalOrderPredicate(n.leader, n.zxid, n.peerEpoch,_{color}
{color:#0747a6} _proposedLeader, proposedZxid, proposedEpoch))_{color}
{color:#0747a6}_{ recvqueue.put(n); break; }_{color}
{color:#0747a6}_}_{color}
So my question is :
Should SID 3 come out of FLE (as LEADING) without waiting long enough to
1. let SID 1 and SID 2 to change their vote to SID 4 (?)
Had SID 3 waited long enough, it SID 3 would have not selected itself as
prospective leader. Instead would have learned that SID 1 and 2 have changed
their votes to make SID 4 as the leader. And meantime would have also obtained
the Notification from SID 4.
So if I increase _finalizeWait_ as a dirty hack/workaround _,_ would it solve
this problem_?_
OR
Somehow, SID 4 and SID 1 should have learned that 1 and 3 are out of election
and instead of verifying that a majority are following SID 3, should have
somehow learn to break a tie(?)
> Leadership Election gets stuck in 5 node ensemble
> -------------------------------------------------
>
> Key: ZOOKEEPER-3707
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3707
> Project: ZooKeeper
> Issue Type: Bug
> Components: leaderElection
> Affects Versions: 3.5.5
> Reporter: Suhas Dantkale
> Priority: Major
>
> Scenario:
> 1. 5 node ensemble-(SID 1,2,3,4,5). 5 is the current Leader.
> 2. Test brings down 5's ZK process.
> 3. Leadership election begins. First each SID votes itself to be the leader
> as expected.
> 4. SID 1 and SID 2 gets notification from SID 3 before they get Notification
> from SID 4. They update their vote to propose 3 as the Leader as expected and
> send notifications.
> 5. SID 3 receives the notification from 1, 2 and itself and its Election
> predicate is successfully terminated and it goes to LEADING state, comes out
> of FLE and goes to the next phase.
> 6. SID 2 meantime goes to FOLLOWING state , comes out of FLE and goes to the
> next phase(NEWLEADER sending etc).
> so far so good.
> 7. Meantime (somewhere after step 4) SID 1 receives notification from SID 4
> and since SID 4 > SID 3(and zxid is same), SID 1 changes its mind and updates
> its proposal - now to elect 4 as leader and sends notification.
> 8. SID 4 is trying to elect itself as leader. And even though SID 2 and SID 3
> are out of election, the SID 4 can not get out of election because - not
> enough number of nodes are following 3(Only 1 is following 3).
> 9. SID 2 is also stuck in FLE like SID 4.
> So, in summary SID 1 and 4 are stuck in FLE (in lookForLeader()) and SID 2
> and SID 3 are stuck in the next phase because SID 3's NEWLEADER is not
> responded by the quorum.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
