Thanks for the replay. >If you are required to see all the deltas you may >purge/merge them only when they are consumed.
Do you mean we can only read all the changes when merging the snapshots? That may be not good enough for HBase because we would like to know all the assignment states when HMaster get the notification from zk. We know the zk pattern and the way to use it. But now zk is not suitable for "state-machine" applications, right? ________________________________________ From: Thawan Kooburat [[email protected]] Sent: Saturday, January 25, 2014 10:49 AM To: [email protected] Subject: Re: Where are we in ZOOKEEPER-1416 Sorry for responding very late to this thread. Seem like it is already settle in some way, so feel free to ignore. I just need add some more detail about this JIRA given that I am its owner now. >My take is that persistent subscriptions add complexity and are not >strictly >necessary. You can follow this pattern of setting a watch, reading the >state >upon a notification and setting a new watch. Why do you feel that's >complicated? The main motivation behind ZK-1416 is to reduce memory footprint required to maintain large number of watches on both the client and the server-side for our internal service discovery. The secondary goal is to simplify the user cases where client have to subscribe too all the data under a given subtree. We have a working implementation in our branch but we haven't used in production yet. The memory issue was partially solved by ZOOKEEPER-1177. And for the second part, existing watch API can be used but it take us a while to get rid of all corner cases. > >Having the ability to know exact deltas would help make HBase region >assignment more robust. > Our service discovery system use deltas to make it efficient to publish information to clients. So we store delta as a new znode and periodically merging into a snapshot. If our client fall to far behind, it can always read snapshot. If you are required to see all the deltas you may purge/merge them only when they are consumed.
