hanm commented on pull request #1690: URL: https://github.com/apache/zookeeper/pull/1690#issuecomment-852694844
> > 1. At line 630 in function 'syncWithLeader' In [Learner.java](https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Learner.java), I see follower in phase SYNCHRONIZATION may receive PROPOSAL and COMMIT. But in paper, leader will add a new server into its 'forwardingFollowers'(Q in paper) only when receiving ack of NEWLEADER from it, so in paper this new server should not receive PROPOSAL or COMMIT because in phase SYNCHRONIZATION it has not sent ACK-NEWLEADER. > So here, my question is in ZAB 1.0, when does leader add a newly joined server into its 'forwardingFollowers'?(I am sorry that I didn't find this in code) You can't find one, because there is no such thing in production. In production, a newly joined server will connect to leader first, the leader will spin up a dedicated handler thread to sync this follower. You can check LearnerHandler implementation for details. The other important detail here is - in paper, the NEWLEADER does not have the same semantic as the actual production implementation. In paper, the NEWLEADER(e, Ie) carries the epoch AND the leader history, but the actual implementation of NEWLEADER is a pure control operation rather than a mixed control + data operation as described in paper. > 2. My understanding of question above is: in phase 2, leader syncs with each follower and ''sends'' NEWLEADER only to this corresponding follower that has completed sync, not ''broadcasts'' NEWLEADER. So when leader is in phase 3, there may exist some follower in phase 2, and has not completed sync with leader since it hasn't received NEWLEADER, but it is allowed to receive PROPOSAL and COMMIT. As previously mentioned, because each follower sync is handled by a dedicated handler thread, there is no such state as "leader is in phase X". We can say that the "handler of follower X is in phase Y", though. Because of this, we don't need worry about "So when leader is in phase 3, there may exist some follower in phase 2", which will not happen, because its the handler thread, instead of leader, that's in different phases; and because one to one mapping between handler thread and follower, their phase always match. > I am not sure whether my understanding is right. If so, I think my spec about phase 2 is that, for each connected follower: first, leader syncs with the follower by an abstract action RECOVERY_SYNC; second, in the action, follower will have the latest history of leader, and leader sends NEWLEADER to it; then, leader and follower communicate using ACK-NEWLEADER and UPTODATE and follower turns to phase 3. Leader turns to phase 3 upon receiving ACK-NEWLEADER from a quorum. The sequences of events is correct. > 3. At line 766 in function 'syncWithLeader' in [Learner.java](https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/Learner.java), I see when receiving NEWLEADER, the variable 'packetsNotCommitted' is set as empty, and when receiving UPTODATE, there is no action handling transactions left to commit them. When follower receives NEWLEADER, it will apply all previous queued proposals from leader atomically to its state machine. The UPTODATE is merely a control operation to signal follower that the leadership and quorum is established, and this specific follower can start serving traffic. > Does that mean follower thinks all transactions committed upon receiving NEWLEADER not UPTODATE? And it seems like that NEWLEADER, ACK-LEADER, UPTODATE carries little data except epoch, just used for acknowledgement between leader and followers. I don't quite get your first question here. The remaining part is correct, these are control operations that mark the milestones of the sync phase, and they don't carry replicated data. > 4. In phase 1 in manual tha we mentioned above, there's one word: "Note, if a follower is connecting, but the leader is already established (in phase 3) the follower follows the phases, but the leader ignores any ACKs." > It seems like that leader in phase 3(BROADCAST) will ignre ACKEPOCH in phase1 and ACK of NEWLEADER in phase 2, then how dose this newly connecting follower receive UPTODATE to go to phase BROADCAST? As previously mentioned, it's the handler thread in leader that handles the sync phase. Each handler thread will have different state but this state will be kept in sync with the corresponding follower where the handler thread is designated to. > > Since these details are related to how to let follower sync with leader in my spec of ZAB 1.0, I want to have a further understanding of them. Thank you! I would recommend read https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zab1.0 - which is exactly the same as production implementation. Maybe treat this as source of truth and use the paper as advisory only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
