Both of the options that Scott mentioned are quite interesting. Quite a few of our users are interested in these two features. I think for 2, we should be able to use observers with a subscription to the master cluster with interested in a special subtree. That avoids too much of cross talk.
Henry/Flavio, do you guys want to keep this in mind for Observers in (not implement it in the jira but to generalize it in a way that partial subscription can be done later) http://issues.apache.org/jira/browse/ZOOKEEPER-368 Wherein an observer can just register with interest in a subtree and the master cluster can avoid sending updates to zookeeper tree not in the sbutree. This would be very helpful in a WAN setting where in only small data needs to be up to date within different data centers. thanks mahadev On 7/20/09 11:50 AM, "Todd Greenwood" <to...@audiencescience.com> wrote: > Flavio, Ted, Henry, Scott, this would perfectly well for my use case > provided: > > SINGLE ENSEMBLE: > GROUP A : ZK Servers w/ read/write AND Leader Elections > GROUP B : ZK Servers w/ read/write W/O Leader Elections > > So, we can craft this via Observers and Hiererarchial Quorum groups? > Great. Problem solved. > > When will this be production ready? :o) > > -------------------- > > Scott brought up a multi-feature that is very interesting for me. > Namely: > > 1. Offline ZK servers that sync & merge on reconnect > > The offline servers seems conceptually simple, it's kind of like a > messaging system. However, the merge and resolve step when two servers > reconnect might be challenging. Cool idea though. > > 2. Partial memory graph subscriptions > > The second idea is partial memory graph subscriptions. This would enable > virtual ensembles to interract on the same physical ensemble. For my use > case, this would prevent unnecessary cross talk between nodes on a WAN, > allowing me to define the subsets of the memory graph that need to be > replicated, and to whom. This would be a huge scalability win for WAN > use cases. > > -Todd > > -----Original Message----- > From: Scott Carey [mailto:sc...@richrelevance.com] > Sent: Monday, July 20, 2009 11:00 AM > To: firstname.lastname@example.org > Subject: Re: Leader Elections > > Observers would be awesome especially with a couple enhancements / > extensions: > > An option for the observers to enter a special state if the WAN link > goes down to the "master" cluster. A read-only option would be great. > However, allowing certain types of writes to continue on a limited basis > would be highly valuable as well. An observer could "own" a special > node and its subnodes. Only these subnodes would be writable by the > observer when there was a session break to the master cluster, and the > master cluster would take all the changes when the link is > reestablished. Essentially, it is a portion of the hierarchy that is > writable only by a specitfic observer, and read-only for others. > The purpose of this would be for when the WAN link goes down to the > "master" ZKs for certain types of use cases - status updates or other > changes local to the observer that are strictly read-only outside the > Observer's 'realm'. > > > On 7/19/09 12:16 PM, "Henry Robinson" <he...@cloudera.com> wrote: > > You can. See ZOOKEEPER-368 - at first glance it sounds like observers > will > be a good fit for your requirements. > > Do bear in mind that the patch on the jira is only for discussion > purposes; > I would not consider it currently fit for production use. I hope to put > up a > much better patch this week. > > Henry > > On Sat, Jul 18, 2009 at 7:38 PM, Ted Dunning <ted.dunn...@gmail.com> > wrote: > >> Can you submit updates via an observer? >> >> On Sat, Jul 18, 2009 at 6:38 AM, Flavio Junqueira <f...@yahoo-inc.com> >> wrote: >> >>> 2- Observers: you could have one computing center containing an > ensemble >>> and observers around the edge just learning committed values. >> >> >> >> >> -- >> Ted Dunning, CTO >> DeepDyve >> >