Leader Elections

2009-07-17 Thread Todd Greenwood
Zookeeper-Users, I'm configuring a zookeeper ensemble that will have servers across a WAN. However, I'd like to constrain the leader elections to elected leaders only from a pool of zookeeper servers in a centralized computing center. In this scenario, zookeeper servers at the edge of the WAN can

RE: Leader Elections

2009-07-20 Thread Todd Greenwood
Flavio, Ted, Henry, Scott, this would perfectly well for my use case provided: SINGLE ENSEMBLE: GROUP A : ZK Servers w/ read/write AND Leader Elections GROUP B : ZK Servers w/ read/write W/O Leader Elections So, we can craft this via Observers and Hiererarchial Quorum groups? Grea

RE: Leader Elections

2009-07-20 Thread Todd Greenwood
ul 20, 2009 at 7:50 PM, Todd Greenwood wrote: > Flavio, Ted, Henry, Scott, this would perfectly well for my use case > provided: > > SINGLE ENSEMBLE: >GROUP A : ZK Servers w/ read/write AND Leader Elections >GROUP B : ZK Servers w/ read/write W/O Leader Elections &g

Zookeeper WAN Configuration

2009-07-24 Thread Todd Greenwood
Like most folks, our WAN is composed of various zones, some central processing, some edge, some corp, and some in between (DMZs). In this model, a given Zookeeper server will not have direct connectivity to all of it's peers in the ensemble due to various security constraints. Is this a problem? Ar

RE: Zookeeper WAN Configuration

2009-07-24 Thread Todd Greenwood
sn't what you want (at all). The ideas for federating ZK or allowing observers would likely do what you want. I can imagine that an observer would only care that it can see it's local peers and one of the observers would be elected to get updates (and thus would care about the central ser

RE: Zookeeper WAN Configuration

2009-07-24 Thread Todd Greenwood
observations: On Jul 24, 2009, at 4:40 PM, Ted Dunning wrote: > On Fri, Jul 24, 2009 at 4:23 PM, Todd Greenwood > wrote: > >> Could you explain the idea behind the Observers feature, what this >> concept is supposed to address, and how it applies to the WAN >> configurat

RE: Zookeeper WAN Configuration

2009-07-26 Thread Todd Greenwood
e.org/zookeeper/docs/r3.2.0/zookeeperAdmin.html Check under "Cluster Options" options like group and weight. -Flavio On Jul 24, 2009, at 5:03 PM, Todd Greenwood wrote: > > In the future, once the Observers feature is implemented, then we > should > be able to deploy zk se

RE: Zookeeper WAN Configuration

2009-07-27 Thread Todd Greenwood
Flavio, more questions inline: -Original Message- From: Flavio Junqueira [mailto:f...@yahoo-inc.com] Sent: Sunday, July 26, 2009 12:49 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Zookeeper WAN Configuration Todd, Answers inline: On Jul 26, 2009, at 11:05 AM, Todd Greenwood

RE: Zookeeper WAN Configuration

2009-07-29 Thread Todd Greenwood
wrote: > Todd, Some more answers. Please check out carefully the information at > the bottom of this message. > > On Jul 27, 2009, at 4:02 PM, Todd Greenwood wrote: > >> >> I'm assuming that you're setting the weight of ZooKeeper servers in >> PODs to

RE: Zookeeper WAN Configuration

2009-07-30 Thread Todd Greenwood
Patrick - Thank you, I'll proceed accordingly. -Todd -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Wednesday, July 29, 2009 10:30 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Zookeeper WAN Configuration > [Todd] What is the recommended policy regarding pat

bad svn url : test-patch

2009-07-30 Thread Todd Greenwood
FYI - looks like there is a bad url in svn... $ svn co http://svn.apache.org/repos/asf/hadoop/zookeeper/branches/branch-3.2 branch-3.2 ... Abranch-3.2/build.xml Fetching external item into 'branch-3.2/src/java/test/bin' svn: URL 'http://svn.apache.org/repos/asf/hadoop/common/nightly/test-pat

RE: bad svn url : test-patch

2009-07-30 Thread Todd Greenwood
of the hadoop split. This file is used for hudson test environment. It isnt used anywhere else, so the svn co otherwise should be fine. We should fix it anyways. Thanks mahadev On 7/30/09 2:57 PM, "Todd Greenwood" wrote: > FYI - looks like there is a bad url in svn... > &

RE: test failures in branch-3.2

2009-07-30 Thread Todd Greenwood
https://issues.apache.org/jira/browse/ZOOKEEPER-492 Patrick Patrick Hunt wrote: > Todd Greenwood wrote: >> The build succeeds, but not the all of the tests. In previous test runs, >> I noticed an error in org.apache.zookeeper.test.FLETest. It was not able >> to bind to a port

RE: test failures in branch-3.2

2009-07-30 Thread Todd Greenwood
Patrick, inline. -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Thursday, July 30, 2009 9:13 PM To: zookeeper-user@hadoop.apache.org Subject: Re: test failures in branch-3.2 Todd Greenwood wrote: > The build succeeds, but not the all of the tests. In previous t

RE: test failures in branch-3.2

2009-07-30 Thread Todd Greenwood
ted abnormally. Please note the time in the report does not reflect the time until the VM exit. -Todd -Original Message- From: Patrick Hunt [mailto:ph...@apache.org] Sent: Thursday, July 30, 2009 10:13 PM To: zookeeper-user@hadoop.apache.org Subject: Re: test failures in branch-3.2 T

RE: test failures in branch-3.2

2009-07-31 Thread Todd Greenwood
Inline. > -Original Message- > From: Patrick Hunt [mailto:ph...@apache.org] > Sent: Thursday, July 30, 2009 10:57 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: test failures in branch-3.2 > > Todd Greenwood wrote: > > Starting w/ branch-3.2 (no ch

RE: test failures in branch-3.2

2009-07-31 Thread Todd Greenwood
er, I'll switch to 3.2.1 as soon as I can. -Todd > -Original Message- > From: Patrick Hunt [mailto:ph...@apache.org] > Sent: Friday, July 31, 2009 11:38 AM > To: zookeeper-user@hadoop.apache.org; Todd Greenwood > Subject: Re: test failures in branch-3.2 > > Hi T

Unending Leader Elections in WAN deploy

2009-07-31 Thread Todd Greenwood
This repro's in both branch-3.2, and branch-3.2+patches(473, 479, 481). Basically, it seems like the nodes are electing pd4-zook02 to be the leader. However, pd4-zook02 seems to realize it's not supposed to be and then disconnects everyone. Then they re-elect it again, and it loops over and over.

RE: Unending Leader Elections in WAN deploy

2009-07-31 Thread Todd Greenwood
Some how the logs did not attach. Zookeeper logs should be attached. > -Original Message- > From: Todd Greenwood [mailto:to...@audiencescience.com] > Sent: Friday, July 31, 2009 7:15 PM > To: zookeeper-user@hadoop.apache.org > Subject: Unending Leader Elections in WAN d

RE: Unending Leader Elections in WAN deploy

2009-07-31 Thread Todd Greenwood
u're missing 491 from your set of patches. > > -Flavio > > On Jul 31, 2009, at 7:15 PM, Todd Greenwood wrote: > > > This repro's in both branch-3.2, and branch-3.2+patches(473, 479, > > 481). > > > > Basically, it seems like the nodes are electing

RE: Unending Leader Elections in WAN deploy

2009-07-31 Thread Todd Greenwood
-Original Message- > > From: Flavio Junqueira [mailto:f...@yahoo-inc.com] > > Sent: Friday, July 31, 2009 7:18 PM > > To: zookeeper-user@hadoop.apache.org > > Subject: Re: Unending Leader Elections in WAN deploy > > > > You're missing 491 from

RE: Unending Leader Elections in WAN deploy

2009-07-31 Thread Todd Greenwood
8 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: Unending Leader Elections in WAN deploy > > It should be in 479. Perhaps you have a stale version of the patch. > > -Flavio > > On Jul 31, 2009, at 7:46 PM, Todd Greenwood wrote: > > > Flavio, > > &

RE: Unending Leader Elections in WAN deploy

2009-08-03 Thread Todd Greenwood
ge- > From: Flavio Junqueira [mailto:f...@yahoo-inc.com] > Sent: Friday, July 31, 2009 9:51 PM > To: zookeeper-user@hadoop.apache.org > Subject: Re: Unending Leader Elections in WAN deploy > > Perfect! Thanks for the update, Todd. > > -Flavio > > On Ju

RE: Unending Leader Elections in WAN deploy

2009-08-03 Thread Todd Greenwood
tty simple: 1. copy the branch-3.2 source to a temp directory (src/patched/branch-3.2) 2. apply the ZOOKEEPER patches in my patches directory 3. build zookeeper in the temp directory -Todd > -Original Message- > From: Todd Greenwood [mailto:to...@audiencescience.com] > Sent: Mond

RE: Unending Leader Elections in WAN deploy

2009-08-03 Thread Todd Greenwood
the patches that you mention should be in the branch 3.2 by tomm > or so. 481, 479 are already in. 480 and 491 should be in by tomm. Would > that > suffice for you? > > Thanks > mahadev > > > On 8/3/09 4:21 PM, "Todd Greenwood" wrote: > > > Another

RE: Unending Leader Elections in WAN deploy

2009-08-04 Thread Todd Greenwood
ctions in WAN deploy > > Hi Todd, > I just committed 480 and 491. You can checkout the 3.2 branch now. > > Thanks > mahadev > > > On 8/3/09 4:29 PM, "Todd Greenwood" wrote: > > > That'd be perfect. Thanks! > > > >> -Original

Zookeeper Client

2009-08-04 Thread Todd Greenwood
Is org.apache.zookeeper.ZooKeeper thread safe? I've started walking through the code to check for mutability, and although the first level children are protected, I haven't fully walked the graph. Perhaps I should ask, is it supposed to be thread safe? -Todd

Runtime Interrogation of the Ensemble

2009-08-31 Thread Todd Greenwood
Is there a way to query an ensemble and find out who is the current leader? Along those lines, what facilities exist for interrogating the ensemble and the ensemble members? I'm thinking (dreaming) of an API like: Ensemble : object representing a zookeeper ensemble Long Ensemble.getLastT

ACL question w/ Zookeeper 3.1.1

2009-09-17 Thread Todd Greenwood
I'm attempting to secure a zookeeper installation using zookeeper ACLs. However, I'm finding that while Ids.OPEN_ACL_UNSAFE works great, my attempts at using Ids.CREATOR_ALL_ACL are failing. Here's a code snippet: public class ZooWrapper { /* 1. Here I'm setting up my authentication. I've got an

RE: ACL question w/ Zookeeper 3.1.1

2009-09-18 Thread Todd Greenwood
stings, as well). Appologies for the cross post. -Todd > -Original Message- > From: Patrick Hunt [mailto:ph...@apache.org] > Sent: Friday, September 18, 2009 11:19 AM > To: zookeeper-...@hadoop.apache.org; zookeeper-user@hadoop.apache.org > Cc: Todd Greenwood > Subject: Re

RE: ACL question w/ Zookeeper 3.1.1

2009-09-21 Thread Todd Greenwood
bj...@1409} lastZxid = 2 xid = 3 response = {org.apache.zookeeper.proto.createrespo...@1360}"'/ACLTest\n" r = {org.apache.zookeeper.proto.replyhea...@1389}"2,2,0\n" xid = 2 zxid = 2 err = 0 request = {org.apache.zookeeper.proto.createrequ...@1355}"'/ACLTest,,v{s{15,s{&

RE: ACL question w/ Zookeeper 3.1.1

2009-09-21 Thread Todd Greenwood
hea...@1445}"0,0,-112\n" v5: > > response = {org.apache.zookeeper.proto.createrespo...@1360}"'/ACLTest\n" > > r = {org.apache.zookeeper.proto.replyhea...@1389}"2,2,0\n" -Todd > -Original Message- > From: Patrick Hunt [mailto:ph...@apache.org] > Sent: M

RE: ACL question w/ Zookeeper 3.1.1

2009-09-21 Thread Todd Greenwood
@apache.org] > Sent: Monday, September 21, 2009 4:32 PM > To: zookeeper-user@hadoop.apache.org; Todd Greenwood > Cc: Patrick Hunt > Subject: Re: ACL question w/ Zookeeper 3.1.1 > > Todd Greenwood wrote: > > Patrick, > > > > Thanks, I'll spend some more time tryi

RE: How do we find the Server the client is connected to?

2009-10-01 Thread Todd Greenwood
Failover testing. > -Original Message- > From: Patrick Hunt [mailto:ph...@apache.org] > Sent: Thursday, October 01, 2009 3:44 PM > To: zookeeper-user@hadoop.apache.org; Rob Baccus > Subject: Re: How do we find the Server the client is connected to? > > That detail is purposefully not expo