[ https://issues.apache.org/jira/browse/ZOOKEEPER-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779194#action_12779194 ]
Mahadev konar commented on ZOOKEEPER-368: ----------------------------------------- thanks henry for the comments and testing! Thanks for all the hard work and responses. I have one more question. Sorry I couldnt find the answer to that, so wanted to ask again. I know looking at the code that this shouldnt be a problem but I think it is worth running a small test for it. - the test is to have 2 servers (s1, S2) from the old code and 1 server (s3) with the new code and verify that s1/s2 or s3 are all capable of becoming the leader and everything works fine with neone of them becoming a leader. This could be done by bringing up s1, s2 and s3 at the same time and killing them one at a time and bringing the other up. SOmething like testing a rolling upgrade wherein one server is the new code and the other servers are old code. This is not testing observers but just testing (though I think it should work fine looking at the code) that the older versions will work with the new version irrespective of which of them is the leader. > Observers: core functionality > ------------------------------ > > Key: ZOOKEEPER-368 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-368 > Project: Zookeeper > Issue Type: New Feature > Components: quorum > Reporter: Flavio Paiva Junqueira > Assignee: Henry Robinson > Fix For: 3.3.0 > > Attachments: obs-refactor.patch, observer-refactor.patch, observers > sync benchmark.png, observers.patch, ZOOKEEPER-368.patch, > ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, > ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, > ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch > > > Edit (Henry Robinson/henryr) 12/11/09: > This JIRA specifically concerns the implementation of non-voting peers called > Observers, their documentation and their tests. > Explicit goals are 1. not breaking any current ZK functionality, 2. enabling > at least one deployment scenario involving Observers, 3. documentation > describing how to use the feature and 4. tests validating the correct > behaviour of 2. > Non goals of this JIRA are 1. performance optimizations specific to > Observers, 2. compatibility with every feature of ZooKeeper (in particular > all leader election protocols), which are both to be addressed in future > JIRAs. > See http://wiki.apache.org/hadoop/ZooKeeper/Observers for more detail of use > cases, proposed design and usage. > See http://wiki.apache.org/hadoop/ZooKeeper/Observers/ReviewGuide for a brief > commentary on the current patch. > ------------- > Currently, all servers of an ensemble participate actively in reaching > agreement on the order of ZooKeeper transactions. That is, all followers > receive proposals, acknowledge them, and receive commit messages from the > leader. A leader issues commit messages once it receives acknowledgments from > a quorum of followers. For cross-colo operation, it would be useful to have a > third role: observer. Using Paxos terminology, observers are similar to > learners. An observer does not participate actively in the agreement step of > the atomic broadcast protocol. Instead, it only commits proposals that have > been accepted by some quorum of followers. > One simple solution to implement observers is to have the leader forwarding > commit messages not only to followers but also to observers, and have > observers applying transactions according to the order followers agreed upon. > In the current implementation of the protocol, however, commit messages do > not carry their corresponding transaction payload because all servers > different from the leader are followers and followers receive such a payload > first through a proposal message. Just forwarding commit messages as they > currently are to an observer consequently is not sufficient. We have a couple > of options: > 1- Include the transaction payload along in commit messages to observers; > 2- Send proposals to observers as well. > Number 2 is simpler to implement because it doesn't require changing the > protocol implementation, but it increases traffic slightly. The performance > impact due to such an increase might be insignificant, though. > For scalability purposes, we may consider having followers also forwarding > commit messages to observers. With this option, observers can connect to > followers, and receive messages from followers. This choice is important to > avoid increasing the load on the leader with the number of observers. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.