Are you saying that in 0.8, producers don't use zkclient? Or don't need it? How can a producer dynamically respond to a change in the kafka cluster without zk?
On Mon, Nov 19, 2012 at 8:07 AM, Jun Rao <jun...@gmail.com> wrote: > Jae, > > In 0.8, producers don't need ZK client anymore. Instead, it uses a new > getMetadata api to get topic/partition/leader information from the broker. > Consumers still need ZK client. We plan to redesign the consumer post 0.8 > and can keep this in mind. > > Thanks, > > Jun > > On Sun, Nov 18, 2012 at 10:35 PM, Bae, Jae Hyeon <metac...@gmail.com> > wrote: > > > I want to suggest kafka should create only one instance of ZkClient > > globally because ZkClient is thread safe and it will make many users > > easily customize kafka source code for Zookeeper. > > > > In our company's cloud environment, it is not recommended to create > > ZkClient from zkConnect string directly because zookeeper cluster can > > be dynamically changing. So, I have to create ZkClient using our > > company's own platform library. Because of this requirement, I can't > > use kafka jar file directly. I can modify and build kafka source code > > but I have to repeat this work whenever I update kafka version, pretty > > annoying. > > > > So, my suggestion is, let me pass ZkClient outs of Producer, Consumer, > > and Broker, as the following example. > > > > Producer<String, String> producer = > > > > > ProducerBuilder.withZkClient(zkClient).build<String,String>(producerConfig); > > > > ConsumerConnector connector = > > Consumer.withZkClient(zkClient).createJavaConsumerConnector(new > > ConsumerConfig(consumerProps)); > > > > KafkaServer is a little more complicated but I believe without much > > effort we can refactor KafkaServer to be customized with ZkClient. > > > > I really appreciate if this suggestion is accepted and merged to 0.8. > > If you want me to contribute with this suggestion, please let me know > > your opinion. If you are positive with this idea, I will contribute > > very happily. > > > > Thank you > > Best, Jae > > >
