Re: Using watcher for being notified of children addition/removal
You may want to consider adding a distributed queue to your use of ZK. As was mentioned previously, watches don't notify you of every change, just that a change was made. For example multiple changes may be "visible" when you get the notification. A distributed queue would allow you to "log" every change, and have your "watcher" process easily process the result. The only issue I could see is one of atomicity, but depending on your use case(s) that may not be an issue, or perhaps one that can be worked around. Patrick On 08/02/2010 09:18 AM, Ted Dunning wrote: Another option besides Steve's excellent one would be to keep something like 1000 nodes in your list per znode. Many update patterns will give you the same number of updates, but the ZK transactions that result (getChildren, read znode) will likely be more efficient, especially the getChildren call. Remember, it is not a requirement that you have a one-to-one mapping between your in-memory objects and in-zookeeper znodes. If that works, fine. If not, feel free to be creative. On Mon, Aug 2, 2010 at 7:45 AM, Steve Gury wrote: Is there any recipe that would provide this feature (or a work around) ?
Re: Using watcher for being notified of children addition/removal
Another option besides Steve's excellent one would be to keep something like 1000 nodes in your list per znode. Many update patterns will give you the same number of updates, but the ZK transactions that result (getChildren, read znode) will likely be more efficient, especially the getChildren call. Remember, it is not a requirement that you have a one-to-one mapping between your in-memory objects and in-zookeeper znodes. If that works, fine. If not, feel free to be creative. On Mon, Aug 2, 2010 at 7:45 AM, Steve Gury wrote: > Is there any recipe that would provide this feature (or a work around) ? >
Re: Using watcher for being notified of children addition/removal
Hi Steve, While it would be theoretically possible to add the name of the node that triggered the event to the watcher notification, the problem is that this may cause you to "miss" events you wanted to see. ZK does not guarantee you will get an event for "every" change. Any changes that occur between the time you get a notification and do a getChildren requesting a new notification do not send notifications. That's why when you get the notification, you must call getChildren and check for ALL changes against your "baseline" since there may be multiple. Obviously with 1m nodes there is a lot of overhead. I would suggest splitting your list into smaller pieces, say 1000 nodes each, and setting a watch on each of them. You mentioned using sequential nodes, which wouldn't work the same way in this case, but either way a redesign is probably in order. -Dave Wright On Mon, Aug 2, 2010 at 10:45 AM, Steve Gury wrote: > Hi, > > I'm interested in being notified by addition/removal of subnode of a > watched node. > > The method getChildren allows to be notified when "a successful > operation that deletes the node of the given path or creates/delete a > child under the node". But this notification only provides the path of > the parent, and I would like to have the path of the added/removed > child. > > For instance: > I create a node "BaseNode", and 1 million of EPHEMERAL_SEQUENTIAL > "ChildNode". I would like to be notified of any new creation/deletion > of "ChildNode" without having to list all "ChildNode" (1 million!) > Ideally, I would like to be notified of any addition/deletion, the > Watcher being called with a WatchedEvent.getPath returning the path of > the ChildNode and not the path of the BaseNode. > > > Is there any recipe that would provide this feature (or a work around) ? > > Thanks, > Steve >
Using watcher for being notified of children addition/removal
Hi, I'm interested in being notified by addition/removal of subnode of a watched node. The method getChildren allows to be notified when "a successful operation that deletes the node of the given path or creates/delete a child under the node". But this notification only provides the path of the parent, and I would like to have the path of the added/removed child. For instance: I create a node "BaseNode", and 1 million of EPHEMERAL_SEQUENTIAL "ChildNode". I would like to be notified of any new creation/deletion of "ChildNode" without having to list all "ChildNode" (1 million!) Ideally, I would like to be notified of any addition/deletion, the Watcher being called with a WatchedEvent.getPath returning the path of the ChildNode and not the path of the BaseNode. Is there any recipe that would provide this feature (or a work around) ? Thanks, Steve