On 7/20/09 2:21 PM, "Scott Carey" <sc...@richrelevance.com> wrote:

> Yes, a more general partial graph subscription / ownership framework would
> allow for not just better WAN scalability but also (and more critically IMO)
> higher reliability.  Often, some large subset of application functionality is
> local to one network, and a minority is global and in need of WAN
> communication.  In this case, when the WAN breaks one wishes that local
> functionality to continue to function, and only those parts truly dependant on
> external events to be interrupted.
> Currently one has to have separate ensembles to partition data and clunky
> 'bridge' code to intercommunicate.
> It would certainly be more natural if two ZK ensembles could register with
> each other, in a 'partial sub-graph publish/subscribe' framework.  It could
> almost be like file system mounting:
> To subscribe:
> subscribe otherEnsemble:port/path/to/otherstuff  /localpath/to/mount/into
> Publishing is the same thing -- think of it as a request for a remote ZK
> cluster to subscribe to the local ZK's data.

Actually, there is one other thing I would add to the above -- subscription
persistence options.  The two basic ones are persistence to disk where the
subgraph survives (read only) if the master is absent -- probably with a
special notification to readers that it is in this state, or it can be
ephemeral and drop if the session to the master breaks.  There are a variety
of options around behavior when the connection to the master breaks.

>> -Todd
>> -----Original Message-----
>> From: Scott Carey [mailto:sc...@richrelevance.com]
>> Sent: Monday, July 20, 2009 11:00 AM
>> To: zookeeper-user@hadoop.apache.org
>> Subject: Re: Leader Elections
>> Observers would be awesome especially with a couple enhancements /
>> extensions:
>> An option for the observers to enter a special state if the WAN link
>> goes down to the "master" cluster.  A read-only option would be great.
>> However, allowing certain types of writes to continue on a limited basis
>> would be highly valuable as well.  An observer could "own" a special
>> node and its subnodes.  Only these subnodes would be writable by the
>> observer when there was a session break to the master cluster, and the
>> master cluster would take all the changes when the link is
>> reestablished.  Essentially, it is a portion of the hierarchy that is
>> writable only by a specitfic observer, and read-only for others.
>> The purpose of this would be for when the WAN link goes down to the
>> "master" ZKs for certain types of use cases - status updates or other
>> changes local to the observer that are strictly read-only outside the
>> Observer's 'realm'.
>> On 7/19/09 12:16 PM, "Henry Robinson" <he...@cloudera.com> wrote:
>> You can. See ZOOKEEPER-368 - at first glance it sounds like observers
>> will
>> be a good fit for your requirements.
>> Do bear in mind that the patch on the jira is only for discussion
>> purposes;
>> I would not consider it currently fit for production use. I hope to put
>> up a
>> much better patch this week.
>> Henry
>> On Sat, Jul 18, 2009 at 7:38 PM, Ted Dunning <ted.dunn...@gmail.com>
>> wrote:
>>> Can you submit updates via an observer?
>>> On Sat, Jul 18, 2009 at 6:38 AM, Flavio Junqueira <f...@yahoo-inc.com>
>>> wrote:
>>>> 2- Observers: you could have one computing center containing an
>> ensemble
>>>> and observers around the edge just learning committed values.
>>> --
>>> Ted Dunning, CTO
>>> DeepDyve

Reply via email to