Hi,
1. ZK is used for co-ordination between brokers, controller election,
leader election, storing topic configuration etc.
I think we use both sequential and ephemeral nodes.
2. Yes, Kafka uses ZK watches for controller changes, new topic creation,
new partition creation, leader change on partition etc. some links here:
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/ZookeeperTopicEventWatcher.scala
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala
3. Kafka data structures in ZK are documented here:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper
Kumar
On Tue, Aug 18, 2015 at 6:33 PM, Prabhjot Bharaj <[email protected]>
wrote:
> Hello Folks,
>
> I wish to contribute to Kafka internals. And, one of the things which can
> help me do that is understanding how kafka uses zookeeper. I have some of
> these basic doubts:-
>
> 1. Is zookeeper primarily used for locking ? If yes, in what cases and what
> kind of nodes does it use - sequential/ephemeral?
>
> 2. Does kafka use zookeeper watches for any of functions ?
>
> 3. What kind of state is stored in Zookeeper ? (I believe it has to be the
> leader information per partition, but is there anything apart from it?)
> What is the scale of data that is stored in Zookeeper ?
>
> Looking forward for your help.
>
> Thanks,
> prabcs
>
