Flavio, thank you for the suggestion.

I have looked at the documention (relevant snippets pasted in below), and 
looked at the presentations 
but I still have some questions about WAN configuration:

A <-> B
A <-> C
A <-> D

A is a central processing hub (DC).
B-D are remote colo edge nodes (PODS).
Each POD contains (m) ZK Servers with (q) client connections.
What are the advantages and disadvantages to co-locating ZK Servers across a 
WAN? Could you correct my admitedly naïve assumtions here?

1. ZK Servers within a POD would significantly improve read/write performance 
within a given POD, v.s. clients within the POD opening connections to the DC.

2. ZK Servers within a POD would provide local file transacted storage of 
writes, obviating the need to write that code ourselves.

3. ZK Servers within the POD would be resilient to network connectivity failure 
between the POD and the DC. Once connectivity re-established, the ZK Servers in 
the POD would sync with the ZK servers in the DC, and, from the perspective of 
a client within the POD, everything just worked, and there was no network 

4. A WAN topology of co-located ZK servers in both the DC and (n) PODs would 
not significantly degrade the performance of the ensemble, provided large blobs 
of traffic were not being sent across the network.

Doc references below



    (No Java system property)

    Enables a hierarchical quorum construction."x" is a group identifier and 
the numbers following the "=" sign correspond to server identifiers. The 
left-hand side of the assignment is a colon-separated list of server 
identifiers. Note that groups must be disjoint and the union of all groups must 
be the ZooKeeper ensemble.

    (No Java system property)

    Used along with "group", it assigns a weight to a server when forming 
quorums. Such a value corresponds to the weight of a server when voting. There 
are a few parts of ZooKeeper that require voting such as leader election and 
the atomic broadcast protocol. By default the weight of server is 1. If the 
configuration defines groups, but not weights, then a value of 1 will be 
assigned to all servers.


A different construction that uses weights and is useful in wide-area 
deployments (co-locations) is a hierarchical one. With this construction, we 
split the servers into disjoint groups and assign weights to processes. To form 
a quorum, we have to get a hold of enough servers from a majority of groups G, 
such that for each group g in G, the sum of votes from g is larger than half of 
the sum of weights in g. Interestingly, this construction enables smaller 
quorums. If we have, for example, 9 servers, we split them into 3 groups, and 
assign a weight of 1 to each server, then we are able to form quorums of size 
4. Note that two subsets of processes composed each of a majority of servers 
from each of a majority of groups necessarily have a non-empty intersection. It 
is reasonable to expect that a majority of co-locations will have a majority of 
servers available with high probability.

With ZooKeeper, we provide a user with the ability of configuring servers to 
use majority quorums, weights, or a hierarchy of groups.

-----Original Message-----
From: Flavio Junqueira [mailto:f...@yahoo-inc.com] 
Sent: Saturday, July 25, 2009 7:55 AM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Zookeeper WAN Configuration

Todd, you can try using flexible quorums to implementing what your  
requesting. You can simulate the behavior I described of observers by  
setting the weight of the server to zero. Please check the  
documentation at:


Check under "Cluster Options" options like group and weight.


On Jul 24, 2009, at 5:03 PM, Todd Greenwood wrote:

> In the future, once the Observers feature is implemented, then we  
> should
> be able to deploy zk servers to both the DC and to the pods...with all
> the goodness that Flavio mentions below.
> -----Original Message-----
> From: Flavio Junqueira [mailto:f...@yahoo-inc.com]
> Sent: Friday, July 24, 2009 4:50 PM
> To: zookeeper-user@hadoop.apache.org
> Subject: Re: Zookeeper WAN Configuration
> Just a few quick observations:
> On Jul 24, 2009, at 4:40 PM, Ted Dunning wrote:
>> On Fri, Jul 24, 2009 at 4:23 PM, Todd Greenwood
>> <to...@audiencescience.com>wrote:
>>> Could you explain the idea behind the Observers feature, what this
>>> concept is supposed to address, and how it applies to the WAN
>>> configuration problem in particular?
>> Not really.  I am just echoing comments on observers from them that
>> know.
> Without observers, increasing the number of servers in an ensemble
> enables higher read throughput, but causes write throughput to drop
> because the number of votes to order each write operation increases.
> Essentially, observers are zookeeper servers that don't vote when
> ordering updates to the zookeeper state. Adding observers enables
> higher read throughput affecting minimally write throughput (leader
> still has to send commits to everyone, at least in the version we have
> been working on).
>>> """
>>> The ideas for federating ZK or allowing observers would likely do
>>> what
>>> you
>>> want.  I can imagine that an observer would only care that it can  
>>> see
>>> it's
>>> local peers and one of the observers would be elected to get updates
>>> (and
>>> thus would care about the central service).
>>> """
>>> This certainly sounds like exactly what I want...Was this
>>> introduced in
>>> 3.2 in full, or only partially?
>> I don't think it is even in trunk yet.  Look on Jira or at the
>> recent logs
>> of this mailing list.
> It is not on trunk yet.
> -Flavio

Reply via email to