I think its not a good idea to depend precise deltas, it is very difficult to program correctly and handle all corner cases. For example, what happens if the hbase master fails and by the time stand by becomes master there were other changes on ZK. The new Master will miss the changes that happened.
Probably a better approach is to always depend on the snapshot of the cluster state. The function that handles any event should be idempotent given the state of the cluster. On Fri, Jan 17, 2014 at 12:34 PM, Ted Yu <[email protected]> wrote: > HBASE-5487 is also related. > > The discussion there is very long. Below is an excerpt from Honghua: > > too many tricky scenarios/bugs due to ZK watch is one-time(which can result > in missed state transition) and the notification/process is > asyncronous(which can lead to delayed/non-update-to-date state in master > memory). > > Cheers > > > On Fri, Jan 17, 2014 at 11:25 AM, Ted Yu <[email protected]> wrote: > > > Hi, Flavio: > > HBASE-8365 is one such case. > > > > Let me search around for other related discussion. > > > > > > On Fri, Jan 17, 2014 at 11:17 AM, Flavio Junqueira < > [email protected]>wrote: > > > >> Hi Ted, > >> > >> Can you provide more detail on how the precise deltas could make it more > >> robust? > >> > >> -Flavio > >> > >> -----Original Message----- > >> From: "Ted Yu" <[email protected]> > >> Sent: 17/01/2014 17:25 > >> To: "[email protected]" <[email protected]> > >> Subject: Re: Where are we in ZOOKEEPER-1416 > >> > >> Having the ability to know exact deltas would help make HBase region > >> assignment more robust. > >> > >> Cheers > >> > >> > >> > >> On Fri, Jan 17, 2014 at 9:13 AM, kishore g <[email protected]> wrote: > >> > >> > I agree with you, I like the side effect and in fact I would prefer to > >> have > >> > one notification for all changes under a parent node. > >> > > >> > However, Hao is probably asking for ability to know exact deltas. > >> > > >> > > >> > On Fri, Jan 17, 2014 at 8:15 AM, FPJ <[email protected]> wrote: > >> > > >> > > We don't need to have a mapping between every change and a > >> notification. > >> > If > >> > > there are 2+ changes between notifications, you'll be able to > observe > >> it > >> > by > >> > > reading the ZK state. In fact, one nice side-effect is that we > reduce > >> the > >> > > number of notifications when there are many concurrent changes. > >> > > > >> > > The only situation I can see it being necessary is the one in which > we > >> > need > >> > > to know precisely the changes and we haven't cached a previous > >> version of > >> > > the state. > >> > > > >> > > -Flavio > >> > > > >> > > > -----Original Message----- > >> > > > From: kishore g [mailto:[email protected]] > >> > > > Sent: 17 January 2014 16:06 > >> > > > To: [email protected] > >> > > > Subject: Re: Where are we in ZOOKEEPER-1416 > >> > > > > >> > > > I think Hao is pointing out that there is no way to see every > change > >> > > > (delta) that happened to a znode. Consider 2 changes A,B in quick > >> > > > succession. When client gets notified of A and before setting the > >> watch > >> > > the > >> > > > change B has occurred on the server side. This means the client > >> cannot > >> > > know > >> > > > the delta A. Client can only read the state after change B is > >> applied. > >> > > > > >> > > > Implementing the concept of Persistent watcher guarantees that > >> client > >> > if > >> > > > notified after every change. > >> > > > > >> > > > This is a nice to have feature but I dont understand the > >> requirement in > >> > > Hbase > >> > > > where this is needed. Hao, can you shed more light on how this > >> would be > >> > > > useful for HBase (to act like state machine) > >> > > > > >> > > > thanks, > >> > > > Kishore G > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > On Fri, Jan 17, 2014 at 5:18 AM, FPJ <[email protected]> > wrote: > >> > > > > >> > > > > But you don't really miss events, you'll see them when you read > >> the > >> > ZK > >> > > > > state. If you follow the pattern I described, you're supposed to > >> > > > > observe all changes. Perhaps I'm missing some concrete use case > >> you > >> > > > > have mind. > >> > > > > > >> > > > > -Flavio > >> > > > > > >> > > > > > -----Original Message----- > >> > > > > > From: 陈迪豪 [mailto:[email protected]] > >> > > > > > Sent: 17 January 2014 13:03 > >> > > > > > To: [email protected] > >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416 > >> > > > > > > >> > > > > > No, it's not complicated. But for the people who don't > >> understand > >> > zk > >> > > > > deeply, > >> > > > > > they would easily ignore the fact that they would miss events > in > >> > > > > > some > >> > > > > way. > >> > > > > > Moreover, I think providing persistent watch is good for > >> developers > >> > > > > > to > >> > > > > build > >> > > > > > the "state-machine" application. Actually, HBase suffer from > >> > missing > >> > > > > > the intermediate state when use zk to store the data. > >> > > > > > > >> > > > > > If the feature is implemented, I would like to see the patch > and > >> > > > > > consider > >> > > > > if it > >> > > > > > can be used for us. > >> > > > > > > >> > > > > > ________________________________________ > >> > > > > > From: Flavio Junqueira [[email protected]] > >> > > > > > Sent: Friday, January 17, 2014 8:12 PM > >> > > > > > To: [email protected] > >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416 > >> > > > > > > >> > > > > > My take is that persistent subscriptions add complexity and > are > >> not > >> > > > > strictly > >> > > > > > necessary. You can follow this pattern of setting a watch, > >> reading > >> > > > > > the > >> > > > > state > >> > > > > > upon a notification and setting a new watch. Why do you feel > >> that's > >> > > > > > complicated? > >> > > > > > > >> > > > > > -Flavio > >> > > > > > > >> > > > > > -----Original Message----- > >> > > > > > From: 陈迪豪 [mailto:[email protected]] > >> > > > > > Sent: Friday, January 17, 2014 3:13 AM > >> > > > > > To: [email protected] > >> > > > > > Subject: Where are we in ZOOKEEPER-1416 > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > Persistent watch and implementing the feature to act like > "state > >> > > > machine" > >> > > > > > which is mentioned in > >> > > > > > ZOOKEEPER-153< > https://issues.apache.org/jira/browse/ZOOKEEPER- > >> > > > 153>, > >> > > > > > would be great for ZooKeeper user. I think HBase would like to > >> know > >> > > > > > all > >> > > > > the > >> > > > > > change in zk rather than missing kind of events. > >> > > > > > > >> > > > > > So, would we continue developing these features? It's also a > >> little > >> > > > > > complicated to develop with zk and I think there're lots of > >> things > >> > > > > > to > >> > > > > improve. > >> > > > > > >> > > > > > >> > > > > > >> > > > >> > > > >> > > >> > > > > >
