You know, Curator has the ServiceDiscovery recipe that already does what you describe.
-Jordan On November 3, 2014 at 12:41:34 AM, Tony Jackson ([email protected]) wrote: I have a client (or several clients responsible for different services) watching a particular znode, and will notify other services that newly registered machines are available now. The registered machines may be up to e.g. 4k~5k. So effect like diff would be preferred if possible. Otherwise clients responsible for watching znodes need to cache and diff by itself. That seems to be an overhead for clients because it needs to hold many information related to the registered nodes and perform diff function with old one. However I am naive about this, so I appreciate any advises. Thanks >Sent: Monday, November 03, 2014 at 1:11 PM >From: "Cameron McKenzie" <[email protected]> >To: [email protected] >Subject: Re: Watcher latency question >hey Tony, >The way that watches work in ZK is that once they fire they need to be added >back again. If the data >in ZK changes before the watch is reset, then the >client will not find out about this change in data. >i.e You're watching the data in node /test >-Data changes to state 'A' >-Watch fires >-Data changes to state 'B' >-Data changes to state 'A' >-Reset watch >-Data changes to state 'C' >-Watch fires >You're going to miss the intermediate state where the data transitioned from >state A to state B and >back to state A again. This is just a limitation of >ZK, there's a window of opportunity for these >events to be missed. I don't >think that the PathCache in Curator is going to solve this problem. >Do you have a particular use case where missing these transitions is an issue? > >cheers >Cam >>On Mon, Nov 3, 2014 at 4:04 PM, Tony Jackson <[email protected]> wrote:I >>read some articles mentioning that Zookeeper watcher has latency issue. >>Something like before the next watcher is placed and after the previous >>watcher is triggered, it is possible that the client may not receive >>notification within that period. >>http://www.quora.com/Does-Zookeeper-clients-keep-open-lots-of-TCP-connections-if-so-how-scalable-is-it-Any-limits[http://www.quora.com/Does-Zookeeper-clients-keep-open-lots-of-TCP-connections-if-so-how-scalable-is-it-Any-limits] >> >>Then on the internet some people recommend using curator pathcache, which >>helps watch child znodes being added, updated, etc. operations. So my >>question - is it the right recipe to use if I want to avoid watcher latency >>problem? Otherwise which recipe should I use instead or how to avoid such >>problem with curator? >>http://curator.apache.org/curator-recipes/path-cache.html[http://curator.apache.org/curator-recipes/path-cache.html] >> >>Thanks
