Henry Robinson commented on ZOOKEEPER-368:

Hi Mahadev - 

To answer your question one by one:

* I think the read-scalability issue is an important use case. Observers are 
also a natural way to deal with peers that try to connect but are not part of 
the current view, which will be important for the dynamic membership patch. I 
also would like to use them to publish proposals; for example they are a 
convenient point to integrate a larger publish-subscribe system. Watches are 
very useful for their intended purposes but are not ideal for listening to a 
stream of proposals since they are cancelled once fired. 
* Set peerType=observer in the configuration file to make a peer an observer. 
In the dynamic membership patch, a follower that tries to connect while not in 
the current view will become an observer until the view is changed to include 
* Yes, there have been no invasive changes in the Follower code (but some 
restructuring to allow code re-use). All tests currently pass for me, which 
while not conclusive proof, suggests that there have been no behavioural 
changes and none are in by design. 
* We must test in the same way we test the behaviour of Followers - their 
ability to connect, to see proposals, to withstand stress. Some important 
observer specific test cases will be:
** Ensuring an observer does not vote in an election or in a proposal (by 
testing when an ensemble is not quorate but would be if the observer were a 
** Making sure that an observer does not see messages it shouldn't (PROPOSALs 
in particular)
** Ensuring that an observer does not become a leader.

> Observers
> ---------
>                 Key: ZOOKEEPER-368
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-368
>             Project: Zookeeper
>          Issue Type: New Feature
>          Components: quorum
>            Reporter: Flavio Paiva Junqueira
>            Assignee: Henry Robinson
>         Attachments: ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, 
> ZOOKEEPER-368.patch
> Currently, all servers of an ensemble participate actively in reaching 
> agreement on the order of ZooKeeper transactions. That is, all followers 
> receive proposals, acknowledge them, and receive commit messages from the 
> leader. A leader issues commit messages once it receives acknowledgments from 
> a quorum of followers. For cross-colo operation, it would be useful to have a 
> third role: observer. Using Paxos terminology, observers are similar to 
> learners. An observer does not participate actively in the agreement step of 
> the atomic broadcast protocol. Instead, it only commits proposals that have 
> been accepted by some quorum of followers.
> One simple solution to implement observers is to have the leader forwarding 
> commit messages not only to followers but also to observers, and have 
> observers applying transactions according to the order followers agreed upon. 
> In the current implementation of the protocol, however, commit messages do 
> not carry their corresponding transaction payload because all servers 
> different from the leader are followers and followers receive such a payload 
> first through a proposal message. Just forwarding commit messages as they 
> currently are to an observer consequently is not sufficient. We have a couple 
> of options:
> 1- Include the transaction payload along in commit messages to observers;
> 2- Send proposals to observers as well.
> Number 2 is simpler to implement because it doesn't require changing the 
> protocol implementation, but it increases traffic slightly. The performance 
> impact due to such an increase might be insignificant, though.
> For scalability purposes, we may consider having followers also forwarding 
> commit messages to observers. With this option, observers can connect to 
> followers, and receive messages from followers. This choice is important to 
> avoid increasing the load on the leader with the number of observers. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to