Both of the options that Scott mentioned are quite interesting. Quite a few
of our users are interested in these two features. I think for 2, we should
be able to use observers with a subscription to the master cluster with
interested in a special subtree. That avoids too much of cross talk.
Henry/Flavio, do you guys want to keep this in mind for Observers in (not
implement it in the jira but to generalize it in a way that partial
subscription can be done later)
Wherein an observer can just register with interest in a subtree and the
master cluster can avoid sending updates to zookeeper tree not in the
sbutree. This would be very helpful in a WAN setting where in only small
data needs to be up to date within different data centers.
On 7/20/09 11:50 AM, "Todd Greenwood" <to...@audiencescience.com> wrote:
> Flavio, Ted, Henry, Scott, this would perfectly well for my use case
> SINGLE ENSEMBLE:
> GROUP A : ZK Servers w/ read/write AND Leader Elections
> GROUP B : ZK Servers w/ read/write W/O Leader Elections
> So, we can craft this via Observers and Hiererarchial Quorum groups?
> Great. Problem solved.
> When will this be production ready? :o)
> Scott brought up a multi-feature that is very interesting for me.
> 1. Offline ZK servers that sync & merge on reconnect
> The offline servers seems conceptually simple, it's kind of like a
> messaging system. However, the merge and resolve step when two servers
> reconnect might be challenging. Cool idea though.
> 2. Partial memory graph subscriptions
> The second idea is partial memory graph subscriptions. This would enable
> virtual ensembles to interract on the same physical ensemble. For my use
> case, this would prevent unnecessary cross talk between nodes on a WAN,
> allowing me to define the subsets of the memory graph that need to be
> replicated, and to whom. This would be a huge scalability win for WAN
> use cases.
> -----Original Message-----
> From: Scott Carey [mailto:sc...@richrelevance.com]
> Sent: Monday, July 20, 2009 11:00 AM
> To: email@example.com
> Subject: Re: Leader Elections
> Observers would be awesome especially with a couple enhancements /
> An option for the observers to enter a special state if the WAN link
> goes down to the "master" cluster. A read-only option would be great.
> However, allowing certain types of writes to continue on a limited basis
> would be highly valuable as well. An observer could "own" a special
> node and its subnodes. Only these subnodes would be writable by the
> observer when there was a session break to the master cluster, and the
> master cluster would take all the changes when the link is
> reestablished. Essentially, it is a portion of the hierarchy that is
> writable only by a specitfic observer, and read-only for others.
> The purpose of this would be for when the WAN link goes down to the
> "master" ZKs for certain types of use cases - status updates or other
> changes local to the observer that are strictly read-only outside the
> Observer's 'realm'.
> On 7/19/09 12:16 PM, "Henry Robinson" <he...@cloudera.com> wrote:
> You can. See ZOOKEEPER-368 - at first glance it sounds like observers
> be a good fit for your requirements.
> Do bear in mind that the patch on the jira is only for discussion
> I would not consider it currently fit for production use. I hope to put
> up a
> much better patch this week.
> On Sat, Jul 18, 2009 at 7:38 PM, Ted Dunning <ted.dunn...@gmail.com>
>> Can you submit updates via an observer?
>> On Sat, Jul 18, 2009 at 6:38 AM, Flavio Junqueira <f...@yahoo-inc.com>
>>> 2- Observers: you could have one computing center containing an
>>> and observers around the edge just learning committed values.
>> Ted Dunning, CTO