Hi Li, I think reducing the client's complexity and improving performance were two of the main reasons for the change. The rebalance protocol on top of Zookeeper was difficult to implement correctly, and I think a number of Kafka clients never actually got it working. Removing it as a dependence also makes a lot of sense with the introduction of security in 0.9. Zookeeper is easier to protect since it only needs to be accessed by the brokers and the clients are easier to audit since all requests go through Kafka. The full rationale is documented here: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Client+Re-Design.
-Jason On Sat, Dec 5, 2015 at 1:07 AM, Li Tao <[email protected]> wrote: > Hi guys, > > As we know, the new consumer of 0.9 kafka does not need to communicate with > zookeeper. > What is the motive / rational behind this disign? I have some > guesses(correct me if i am wrong): > > 1. Reduce the complexity of consumer. It's hard to maintain the connections > and states of kafka servers and zookeeper servers. > > 2. Improve performance. Too many consumers querying and writting to > zookeeper will decrease the performance as a whole. > > Thanks. >
