Note that broker metadata is not necessarily kept in sync with zookeeper on all brokers at all times: https://issues.apache.org/jira/browse/KAFKA-1367
This looks like it is fixed in the upcoming 0.8.3 > On Aug 8, 2015, at 01:08, Abdoulaye Diallo <abdoulaye...@gmail.com> wrote: > > @Rahul > >> If this is true, why does the producer API makes it necessary to supply a >> value for metadata.broker.list? > > > I believe this is because of the removal of ZK dependency(load balancing is > no longer achieved with the use of ZK). For that purpose, 0.8 producer relies > on the new cluster metadata api… which requires metadata.broker.list. >> https://issues.apache.org/jira/browse/KAFKA-369 >> <https://issues.apache.org/jira/browse/KAFKA-369> > Abdoulaye >> On Aug 7, 2015, at 9:10 PM, Rahul Jain <rahul...@gmail.com> wrote: >> >>> >>> Alternatively you can get the same metadata from Zookeeper >> >> If this is true, why does the producer API makes it necessary to supply a >> value for metadata.broker.list? >> >> I noticed that this wasn't the case in 0.7. >> >> >> >> On 8 Aug 2015 04:06, "Lukas Steiblys" <lu...@doubledutch.me> wrote: >> >>> Hi Qi, >>> >>> Yes, the metadata request will return information about all the brokers in >>> the cluster. Alternatively you can get the same metadata from Zookeeper. >>> >>> Lukas >>> >>> -----Original Message----- From: Qi Xu >>> Sent: Friday, August 7, 2015 2:30 PM >>> To: users@kafka.apache.org >>> Subject: Kafka metadata >>> >>> Hi Everyone, >>> I have a question that hopes to get some clarification. >>> In a Kafka cluster, does every broker have the complete view of the >>> metadata information? >>> What's the best practice for aproducer to send metadata request? Is it >>> recommended to send it to all brokers or just one broker? >>> >>> In our scenario, we want to avoid the situation that each producer needs to >>> talk to every broker because we have hundreds thousands of producers and >>> the scalability of connection number will be a concern if every cluster >>> node is connected by any producer. >>> >>> Thanks you for your help. >>> >>> Qi >>> >
