Hi, I have checked out the latest code out of https://github.com/apache/kafka based on commit id e582447adb4708731aff74aa294e7ce2b30b0a41. Looks like the performance test on the new-consumer is broken.
in/kafka-consumer-perf-test.sh --zookeeper zkip:2181 --broker-list brokerIp:9092 --topic test --messages 50000 --new-consumer The test does not return any response. Is this expected and is there a better way to test the new-consumer. Thanks, -Poorna On Thu, Aug 27, 2015 at 2:25 PM, Poorna Chandra Tejashvi Reddy < pctre...@gmail.com> wrote: > Hi, > > We have built the latest kafka from https://github.com/apache/kafka based > on this commit id 436b7ddc386eb688ba0f12836710f5e4bcaa06c8 . > We ran the performance test on a 3 node kafka cluster. There is a huge > throughput degradation using the new-consumer compared to the regular > consumer. Below are the numbers that explain the same. > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > brokerIp:9092 --topics test --messages 5000000 : gives a throughput of 693 K > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > brokerIp:9092 --topics test --messages 5000000 --new-consumer : gives a > throughput of 51k > > The whole set up is based on ec2, Kafka brokers running on r3.2x large. > > Are you guys aware of this performance degradation , do you have a JIRA > for this, which can be used to track the resolution. > > > Thanks, > > -Poorna >
