Very useful thread on how watchers work in ZooKeeper and why they work that way!
I talked with a Kafka dev about this. He said there are only talks for now about removing ZooKeeper, not going to happen soon, but some people definitely want it done. They even want their own Leader election, so not just the message/subscribe model. I think this is very dangerous. I like Ted's saying that ZooKeeper is not just battle tested, but battle scarred. Anyway, my takeaway has been said before: Kafka is using watchers as if it were meant to be a messaging service, which it is not. (The already linked ZOOKEEPER-153 is what they want, as Patrick stated). And anyone interested on watchers in ZooKeeper - read this thread! :) Cheers! Norbert On Sat, Aug 10, 2019 at 3:48 AM Ted Dunning <ted.dunn...@gmail.com> wrote: > On Wed, Aug 7, 2019 at 5:55 PM Michael Han <h...@apache.org> wrote: > > > Related discussions: > > https://lists.apache.org/list.html?d...@kafka.apache.org:lte=1M:KIP-500 > > > > >> Why would this lead to any discrepancy? It seems to me that the > > controller, will read an even newer state in such a scenario. > > > > I think in this case the Kafka side of expectation is not just be able to > > get latest state, but also not miss any of the state changes. > > > > > The core of the problem is that if you need to see all the changes, then it > is messages, not state. >
