Hi Poorna, We've been looking at the same issue as part of KAFKA-2489. I suspect the initial rebalance is causing the test to timeout, but we're still investigating.
Thanks, Jason On Wed, Sep 2, 2015 at 2:03 PM, Poorna Chandra Tejashvi Reddy < pctre...@gmail.com> wrote: > Hi, > > I have checked out the latest code out of https://github.com/apache/kafka > based > on commit id e582447adb4708731aff74aa294e7ce2b30b0a41. Looks like the > performance test on the new-consumer is broken. > > in/kafka-consumer-perf-test.sh --zookeeper zkip:2181 --broker-list > brokerIp:9092 --topic test --messages 50000 --new-consumer > > The test does not return any response. Is this expected and is there a > better way to test the new-consumer. > > > Thanks, > > -Poorna > > > On Thu, Aug 27, 2015 at 2:25 PM, Poorna Chandra Tejashvi Reddy < > pctre...@gmail.com> wrote: > > > Hi, > > > > We have built the latest kafka from https://github.com/apache/kafka > based > > on this commit id 436b7ddc386eb688ba0f12836710f5e4bcaa06c8 . > > We ran the performance test on a 3 node kafka cluster. There is a huge > > throughput degradation using the new-consumer compared to the regular > > consumer. Below are the numbers that explain the same. > > > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > > brokerIp:9092 --topics test --messages 5000000 : gives a throughput of > 693 K > > > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > > brokerIp:9092 --topics test --messages 5000000 --new-consumer : gives a > > throughput of 51k > > > > The whole set up is based on ec2, Kafka brokers running on r3.2x large. > > > > Are you guys aware of this performance degradation , do you have a JIRA > > for this, which can be used to track the resolution. > > > > > > Thanks, > > > > -Poorna > > >
