On 7/20/09 2:21 PM, "Scott Carey" <sc...@richrelevance.com> wrote:
> > Yes, a more general partial graph subscription / ownership framework would > allow for not just better WAN scalability but also (and more critically IMO) > higher reliability. Often, some large subset of application functionality is > local to one network, and a minority is global and in need of WAN > communication. In this case, when the WAN breaks one wishes that local > functionality to continue to function, and only those parts truly dependant on > external events to be interrupted. > Currently one has to have separate ensembles to partition data and clunky > 'bridge' code to intercommunicate. > > It would certainly be more natural if two ZK ensembles could register with > each other, in a 'partial sub-graph publish/subscribe' framework. It could > almost be like file system mounting: > To subscribe: > subscribe otherEnsemble:port/path/to/otherstuff /localpath/to/mount/into > > Publishing is the same thing -- think of it as a request for a remote ZK > cluster to subscribe to the local ZK's data. > > Actually, there is one other thing I would add to the above -- subscription persistence options. The two basic ones are persistence to disk where the subgraph survives (read only) if the master is absent -- probably with a special notification to readers that it is in this state, or it can be ephemeral and drop if the session to the master breaks. There are a variety of options around behavior when the connection to the master breaks. > > >> >> -Todd >> >> -----Original Message----- >> From: Scott Carey [mailto:sc...@richrelevance.com] >> Sent: Monday, July 20, 2009 11:00 AM >> To: zookeeper-user@hadoop.apache.org >> Subject: Re: Leader Elections >> >> Observers would be awesome especially with a couple enhancements / >> extensions: >> >> An option for the observers to enter a special state if the WAN link >> goes down to the "master" cluster. A read-only option would be great. >> However, allowing certain types of writes to continue on a limited basis >> would be highly valuable as well. An observer could "own" a special >> node and its subnodes. Only these subnodes would be writable by the >> observer when there was a session break to the master cluster, and the >> master cluster would take all the changes when the link is >> reestablished. Essentially, it is a portion of the hierarchy that is >> writable only by a specitfic observer, and read-only for others. >> The purpose of this would be for when the WAN link goes down to the >> "master" ZKs for certain types of use cases - status updates or other >> changes local to the observer that are strictly read-only outside the >> Observer's 'realm'. >> >> >> On 7/19/09 12:16 PM, "Henry Robinson" <he...@cloudera.com> wrote: >> >> You can. See ZOOKEEPER-368 - at first glance it sounds like observers >> will >> be a good fit for your requirements. >> >> Do bear in mind that the patch on the jira is only for discussion >> purposes; >> I would not consider it currently fit for production use. I hope to put >> up a >> much better patch this week. >> >> Henry >> >> On Sat, Jul 18, 2009 at 7:38 PM, Ted Dunning <ted.dunn...@gmail.com> >> wrote: >> >>> Can you submit updates via an observer? >>> >>> On Sat, Jul 18, 2009 at 6:38 AM, Flavio Junqueira <f...@yahoo-inc.com> >>> wrote: >>> >>>> 2- Observers: you could have one computing center containing an >> ensemble >>>> and observers around the edge just learning committed values. >>> >>> >>> >>> >>> -- >>> Ted Dunning, CTO >>> DeepDyve >>> >> >>
