RE: Zookeeper WAN Configuration

2009-07-30 Thread Todd Greenwood
Patrick - Thank you, I'll proceed accordingly. -Todd -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Wednesday, July 29, 2009 10:30 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Zookeeper WAN Configuration [Todd] What is the recommended policy regarding

Re: Zookeeper WAN Configuration

2009-07-28 Thread Patrick Hunt
Flavio, please enter a doc jira for this if there are no docs, it should be in forrest, not twiki btw. It would be good if you could review the current quorum docs (any type) and create a jira/patch that addresses any/all shortfall. Patrick Flavio Junqueira wrote: Todd, Some more answers.

Re: Zookeeper WAN Configuration

2009-07-26 Thread Ted Dunning
This is the problem. ALL writes go from the leader to all nodes and the transaction isn't done until a quorum of machines have confirmed the write. Unless you have a quorum in the central facility, then all writes be as slow as several round-trips to the peripheral installations. This slows

Zookeeper WAN Configuration

2009-07-24 Thread Todd Greenwood
Like most folks, our WAN is composed of various zones, some central processing, some edge, some corp, and some in between (DMZs). In this model, a given Zookeeper server will not have direct connectivity to all of it's peers in the ensemble due to various security constraints. Is this a problem?

Re: Zookeeper WAN Configuration

2009-07-24 Thread Ted Dunning
Each member needs a connection to a quorum. The quorum is ceiling((N+1) / 2) members of the cluster. This guarantees that network partition does not allow two leaders to go on stamping out revisions independent of each other. On Fri, Jul 24, 2009 at 4:23 PM, Todd Greenwood

Re: Zookeeper WAN Configuration

2009-07-24 Thread Flavio Junqueira
Servers in a quorum need to be able to talk to each other to elect a leader. Once a leader is elected, followers only talk to the leader. Of course, if the leader fails, servers in some quorum will need to talk to each other again. If no quorum can be formed, the system is stalled.

RE: Zookeeper WAN Configuration

2009-07-24 Thread Todd Greenwood
responds to ensemble with update + transaction id -Todd -Original Message- From: Flavio Junqueira [mailto:f...@yahoo-inc.com] Sent: Friday, July 24, 2009 4:50 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Zookeeper WAN Configuration Just a few quick observations: On Jul 24, 2009