Could you try increasing the batch size and use async produce? Thanks, Jun
On Tue, Aug 28, 2012 at 11:39 AM, Rakesh Nair <raknai...@gmail.com> wrote: > I was running couple of performance tests using kafka 0.8. I needed to get > the perf numbers for sync producers with ack's. I am getting data transfer > speeds of around 36MB/sec. But the CPU util is well below 50% and the > IOstat are also normal. Even if i push more data using larger number of > threads, am still getting data transfer speeds of around 30MB/sec. > > What should i do to get better transfer speeds, since obviously the cpu and > io dont seem to be a bottleneck. > Could you tell me what i am doing wrong or are there any configurations > that i need to alter. > > Here is one of the configurations that i had used: > > no. of processors available = 24 > > server.properties configurations: > network.threads=20 > io.threads=20 > partitions = 20 > log.flush.interval=500 > log.defualt.flush.interval=3000 > log.default.flush.scheduler.interval.ms=3000 > > Perf command : ./kafka-producer-perf-test.sh --topic perf_0 --broker-list > localhost:9092 --message-size 1024 --request-num-acks 1 --batch-size 1 > --threads 20 --messages 1000000 > Transfer speed : ~31MB/sec > -- > Regards > Rakesh Nair >
