Hmmm... and what should Kafka do if it wants to see all events? Use RabbitMQ? :) Andor
-----Original Message-----From: Ted Dunning <[email protected]>Repl y-To: [email protected]: [email protected]: Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata QuorumDate: Tue, 13 Aug 2019 09:10:04 -0700 If you want to see all events, use Kafka. On Tue, Aug 13, 2019 at 8:20 AM Jordan Zimmerman < [email protected]>wrote: > Also see https://issues.apache.org/jira/browse/ZOOKEEPER-1416 < > https://issues.apache.org/jira/browse/ZOOKEEPER-1416> > There are many use cases where a client wants to see all events from > agiven parent path down. The semantics of setting one-time watches on > asingle node in ZK are cumbersome for these use cases. FWIW I had a > workingPR a few years ago but it's fallen far behind 3.6 now. > -Jordan > > On Aug 13, 2019, at 8:18 AM, Andor Molnar <[email protected]> wrote: > > Subscriber APIhttps://issues.apache.org/jira/browse/ZOOKEEPER-153 > > > > Is it supposed to be something like a generic Observer API on the > > client > side? > > Observers essentially consume ordered updates of ZAB, so we would > > need > to provide a way for users to implement their own “observers”. They > shouldbe able to filter for path to be more convenient. > > Andor > > > > > > > On 2019. Aug 2., at 20:48, Patrick Hunt <[email protected]> wrote: > > > Michael I think you are describing subscribe - this? > > > https://issues.apache.org/jira/browse/ZOOKEEPER-153 > > > wasn't there some work done to keep tlogs around for a while? Or > > > am I > miss > > > remembering? (fb folks?) > > > I'll also add that we haven't done any benchmarking in quite some > > > time. > It > > > would be interesting to collect a few of these use cases from > > > thecommunity, esp downstreams, and evaluate performance, see if > > > we can > address. > > > Patrick > > > On Fri, Aug 2, 2019 at 11:03 AM Michael Han <[email protected]> > > > wrote: > > > > Folks, > > > > Some of you might already see this. Comments? > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum > > > > What caught my eyes are: > > > > *Worse still, although ZooKeeper is the store of record, the > > > > state inZooKeeper often doesn't match the state that is held in > > > > memory in thecontroller. For example, when a partition leader > > > > changes its ISR in > ZK, > > > > the controller will typically not learn about these changes for > > > > manyseconds. There is no generic way for the controller to > > > > follow theZooKeeper event log. Although the controller can set > > > > one-shot > watches, the > > > > number of watches is limited for performance reasons. When a > > > > watchtriggers, it doesn't tell the controller the current > > > > state-- only that > the > > > > state has changed. By the time the controller re-reads the > > > > znode and > sets > > > > up a new watch, the state may have changed from what it was > > > > when the > watch > > > > originally fired. If there is no watch set, the controller may > > > > not > learn > > > > about the change at all. In some cases, restarting the > > > > controller is > the > > > > only way to resolve the discrepancy.* > > > > I've seen some similar zookeeper use cases that ended up like > > > > what'sdescribed here. How can ZooKeeper solve this? It seems to > > > > me that the > only > > > > solution is to provide linearizable read on watched operations. > Thoughts? > > > > Michael.
