Persistent watches were omitted from ZK on purpose because of the perceived
danger of not have a load shedding mechanism.
Note that when you get a notification, the query you do to get the next
state typically sets the next watch. This guarantees that you don't lose
anything, but it may mean that you effectively get notified of multiple
changes in one notification.
If you really, really want all notifications in order, then what you are
really looking for is a kind of a distributed transaction log. For small
applications, you can implement this by writing logs into ZK znodes. Your
client should remember where they were and as they are notified, they read
the current file to catch up. This has the downside that to update a file
you have to completely rewrite it which makes it inconvenient to put a bunch
of stuff into a single chunk of the log. You would also need a watcher on
the directory to notify you when new log files are created. Aside from the
slightly quadratic update problem, this does what you need.
You can also check out Bookkeeper which is a more scalable distributed
transaction logging system. It addresses the complications of (mis) using
ZK as a log system and attempts to give you a reliable and robust
On Fri, Nov 12, 2010 at 7:16 PM, Chang Song <tru64...@me.com> wrote:
> It's been a couple of weeks since we started integrating zookeeper into our
> Great piece of software. Cut down LOC by more than half.
> Thank you for open sourcing Zookeeper.
> I have a couple of questions, though.
> I was wondering you have any plan to support persistent watch stream
> One-time watch is very inconvenient for us.
> We need to get a stream of notifications in order. We are talking thousands
> of clients.
> Since notification can happen in bulk, we need to set a watch first, and
> once we get a callback,
> we need to check periodically what happens to the children of the watch
> You can consider this as a automatic server-side watch registration
> feature: I'd call this sticky watch.
> I think it is easier to implement this as a sticky watch.
> Another question is if it is possible to coalesce predefined number of
> changes for watch callbacks.
> We have observed that if there are many changes in children node, clients
> gets different number of
> messages in bulks. This makes the architecture of our application from
> event-driven to polling for a while.
> Thus pretty much the same reasoning behind the first question.
> Any comments welcome.
> Thank you.