Thanks Guaozhang.

I have a follow-up question.

Say if I can push 1M/s messages (100 size) into 2 node cluster. Is it safe
to say I should be able to produce more or less 2M/s messages into 4 node
cluster?


On Tue, Jul 8, 2014 at 6:00 PM, Guozhang Wang <wangg...@gmail.com> wrote:

> The second script is using the new producer which allows sending multiple
> in-flight requests whereas the first script use the current (old) producer
> which sends only one request at a time to a certain broker.
>
> The new producer will be officially released in 0.8.2.
>
> Guozhang
>
>
> On Tue, Jul 8, 2014 at 2:22 PM, Chen Song <chen.song...@gmail.com> wrote:
>
> > While testing kafka producer performance, I found 2 testing scripts.
> >
> > 1) performance testing script in kafka distribution
> >
> > bin/kafka-producer-perf-test.sh --broker-list localhost:9092 --messages
> > 10000000 --topic test --threads 10 --message-size 100 --batch-size 10000
> > --compression-codec 1
> >
> > 2) performance testing script mentioned in
> >
> >
> >
> https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines
> >
> > bin/kafka-run-class.sh
> > org.apache.kafka.clients.tools.ProducerPerformance test6 50000000 100
> > -1 acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092
> > buffer.memory=67108864 batch.size=8196
> >
> > based on org.apache.kafka.clients.producer.Producer.
> >
> >
> > On my local testing environment,
> >
> > For 1), the loading is ~15M/s and 150,000 msgs/s for mesage size = 100,
> no
> > matter how I adjust batch size and number of loading threads.
> > For 2), it can load up to 90M/s and 1,000,000 records/s.
> >
> > I am wondering what makes these 2 implementations perform so differently?
> >
> > --
> > Chen Song
> >
>
>
>
> --
> -- Guozhang
>



-- 
Chen Song

Reply via email to