That error indicates the broker closed the connection for some reason. Any useful logs from the broker? It looks like you're using ELB, which could also be the culprit. A connection timeout seems doubtful, but ELB can also close connections for other reasons, like failed health checks.
-Ewen On Tue, Jan 27, 2015 at 11:21 PM, Dillian Murphey <crackshotm...@gmail.com> wrote: > I was running the performance command from a virtual box server, so that > seems like it was part of the problem. I'm getting better results running > this on a server on aws, but that's kind of expected. Can you look at > these results, and comment on the occasional warning I see? I appreciate > it! > > 1220375 records sent, 243928.6 records/sec (23.26 MB/sec), 2111.5 ms avg > latency, 4435.0 max latency. > 1195090 records sent, 239018.0 records/sec (22.79 MB/sec), 2203.1 ms avg > latency, 4595.0 max latency. > 1257165 records sent, 251433.0 records/sec (23.98 MB/sec), 2172.6 ms avg > latency, 4525.0 max latency. > 1230981 records sent, 246196.2 records/sec (23.48 MB/sec), 2173.5 ms avg > latency, 4465.0 max latency. > [2015-01-28 07:19:07,274] WARN Error in I/O with > <myawsloadbalancer>(org.apache.kafka.common.network.Selector) > java.io.EOFException > at > > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) > at org.apache.kafka.common.network.Selector.poll(Selector.java:248) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) > at java.lang.Thread.run(Thread.java:745) > 1090689 records sent, 218137.8 records/sec (20.80 MB/sec), 2413.6 ms avg > latency, 4829.0 max latency. > > On Tue, Jan 27, 2015 at 7:37 PM, Ewen Cheslack-Postava <e...@confluent.io> > wrote: > > > Where are you running ProducerPerformance in relation to ZK and the Kafka > > brokers? You should definitely see much higher performance than this. > > > > A couple of other things I can think of that might be going wrong: Are > all > > your VMs in the same AZ? Are you storing Kafka data in EBS or local > > ephemeral storage? If EBS, have you provisioned enough IOPS. > > > > > > On Tue, Jan 27, 2015 at 4:29 PM, Dillian Murphey < > crackshotm...@gmail.com> > > wrote: > > > > > I'm a new user/admin to kafka. I'm running a 3 node ZK and a 6 brokers > on > > > aws. > > > > > > The performance I'm seeing is shockingly bad. I need some advice! > > > > > > bin/kafka-run-class.sh > org.apache.kafka.clients.tools.ProducerPerformance > > > test2 50000000 100 -1 acks=1 bootstrap.servers=5<myloadbalancer>:9092 > > > buffer.memory=67108864 batch.size=8196 > > > > > > > > > > > > > > > 6097 records sent, 13198.3 records/sec (1.26 MB/sec), 2098.0 ms avg > > > latency, 4306.0 max latency. > > > 71695 records sent, 14339.0 records/sec (1.37 MB/sec), 6658.1 ms avg > > > latency, 9053.0 max latency. > > > 65195 records sent, 13028.6 records/sec (1.24 MB/sec), 11504.0 ms avg > > > latency, 13809.0 max latency. > > > 71955 records sent, 14391.0 records/sec (1.37 MB/sec), 16137.4 ms avg > > > latency, 18541.0 max latency. > > > > > > Thanks for any help! > > > > > > > > > > > -- > > Thanks, > > Ewen > > > -- Thanks, Ewen
