Where are you running ProducerPerformance in relation to ZK and the Kafka brokers? You should definitely see much higher performance than this.
A couple of other things I can think of that might be going wrong: Are all your VMs in the same AZ? Are you storing Kafka data in EBS or local ephemeral storage? If EBS, have you provisioned enough IOPS. On Tue, Jan 27, 2015 at 4:29 PM, Dillian Murphey <crackshotm...@gmail.com> wrote: > I'm a new user/admin to kafka. I'm running a 3 node ZK and a 6 brokers on > aws. > > The performance I'm seeing is shockingly bad. I need some advice! > > bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance > test2 50000000 100 -1 acks=1 bootstrap.servers=5<myloadbalancer>:9092 > buffer.memory=67108864 batch.size=8196 > > > > > 6097 records sent, 13198.3 records/sec (1.26 MB/sec), 2098.0 ms avg > latency, 4306.0 max latency. > 71695 records sent, 14339.0 records/sec (1.37 MB/sec), 6658.1 ms avg > latency, 9053.0 max latency. > 65195 records sent, 13028.6 records/sec (1.24 MB/sec), 11504.0 ms avg > latency, 13809.0 max latency. > 71955 records sent, 14391.0 records/sec (1.37 MB/sec), 16137.4 ms avg > latency, 18541.0 max latency. > > Thanks for any help! > -- Thanks, Ewen
