Inder, Zookeeper is used on the producer side for load balancing. It is used for automatic detection of new and failed brokers.
Thanks, Neha On Sun, Dec 11, 2011 at 2:44 AM, Inder Pall <[email protected]> wrote: > why do u want to integrate zk at producer side. can you please elaborate > your usecase. > > is it for load balancing. > On Dec 7, 2011 12:35 AM, "Jun Rao" <[email protected]> wrote: > > > > David, > > > > Could you also comment whether you want a programming api or a monitoring > > api? > > > > Thanks, > > > > Jun > > > > On Tue, Dec 6, 2011 at 10:03 AM, David Ormsbee <[email protected]> > wrote: > > > > > Jun, Neha: > > > > > > Thank you for pointing me the right way on the Producer issue. :-) > > > > > > As for the metadata requests, I've filed a ticket here: > > > > > > https://issues.apache.org/jira/browse/KAFKA-223 > > > > > > Take care. > > > > > > Dave > > > > > > On Tue, Dec 6, 2011 at 12:05 PM, Neha Narkhede < > [email protected] > > > >wrote: > > > > > > > Hi Dave, > > > > > > > > >> Or should I just make the initial assumption > > > > that all brokers have one partition, send messages to those to create > the > > > > topic on the brokers, and then readjust when the the brokers publish > > > their > > > > partitions-per-topic information to ZK after the initial writes? > > > > > > > > That's exactly what we do. You can take a look at > > > > bootstrapWithExistingBrokers() in > kafka.producer.ZKBrokerPartitionInfo. > > > > > > > > >> As an aside, is there any way to query metadata about the broker > > > > directly? > > > > So that a simple consumer might be able to ask a broker for what > topics > > > > they have, how many partitions, what version it's running, etc? > > > > > > > > Today, we don't have such APIs. But I was thinking we can have > something > > > > similar to the 4-letter commands in zookeeper. > > > > For example, > > > > > > > > echo topics | nc broker-host broker-port (Returns all topics > with > > > > number of partitions on each broker) > > > > echo srvr | nc broker-host broker-port (Returns basic broker > > > stats > > > > like version, number of topics etc) > > > > > > > > If the above functionality is useful, lets file a JIRA for it. > > > > > > > > Thanks > > > > Neha > > > > > > > > On Tue, Dec 6, 2011 at 6:34 AM, David Ormsbee <[email protected]> > > > wrote: > > > > > > > > > Hi folks, > > > > > > > > > > I'm implementing ZooKeeper support in our Python client, and I was > > > > > wondering what the expected behavior of the Producer is when the > > > brokers > > > > > are completely empty. At this point, the brokers have registered > > > > themselves > > > > > in /brokers/ids/[0..N], but there are no topics registered under > > > > > /brokers/topics. If someone wants to create a Producer at this > point > > > and > > > > > start sending messages to a new topic, is that an error because no > > > > brokers > > > > > are registered for that topic? Or should I just make the initial > > > > assumption > > > > > that all brokers have one partition, send messages to those to > create > > > the > > > > > topic on the brokers, and then readjust when the the brokers > publish > > > > their > > > > > partitions-per-topic information to ZK after the initial writes? > > > > > > > > > > I'm sure I'm missing something obvious here, since we'd run into a > > > > similar > > > > > problem for new, empty brokers joining the pool. > > > > > > > > > > As an aside, is there any way to query metadata about the broker > > > > directly? > > > > > So that a simple consumer might be able to ask a broker for what > topics > > > > > they have, how many partitions, what version it's running, etc? Or > is > > > > that > > > > > only ever published through ZooKeeper? > > > > > > > > > > Thank you. > > > > > > > > > > Dave > > > > > > > > > > > > >
