What does CPU look like on the producer? Also, can you please use the apache mailing list (cc'd). This google group is from our pre-apache days and is long dead.
-Jay On Tue, Mar 27, 2012 at 7:42 PM, Kafka <kafka-dev+msga...@googlegroups.com>wrote: > A message has been sent to the > kafka-dev<http://groups.google.com/group/kafka-dev?hl=en_US>group and is > awaiting approval. We've included this message for your review. > > The message requires moderation because the user does not have permission > to post. > > You can approve or > reject<http://groups.google.com/group/kafka-dev/pendmsg?hl=en_US>this message > or you can approve this message by replying to this email. > [image: Visit Google Groups] > > Start <https://groups.google.com/groups/create?hl=en_US&src=email> your > own group, visit <http://groups.google.com/support/?hl=en_US&src=email>the > help center, or > report <https://groups.google.com/groups/abuse?hl=en_US&src=email> abuse. > > > ---------- Forwarded message ---------- > From: 刘明敏 <diveintotomor...@gmail.com> > To: kafka-users@incubator.apache.org > Cc: kafka-...@googlegroups.com > Date: Wed, 28 Mar 2012 10:42:11 +0800 > Subject: Re: [kafka-dev] performance of async producer stops increase when > broker is more than 2 > I have 1 producer on 1 machine. > > The perf of sync producer is : > > 1 borker: > *20.3277（M/s）*,*213151.4441(#/s)* (sync) > 2 borkers: > *39.5995*, *415230.6606* (sync) > 3 borkers: > *58.0836*, *609050.4903*(sync) > > compared with async producer : > when 1 broker: > 20.3164 M/s,213033.3823#/s (async) > > when 2 brokers: > 37.4754 M/s,392958.1892#/s (async) > > when 3 brokers(perf stops increase) > 36.6925 M/s, 384748.5668#/s (async) > > the sync producer reached 600K #/s(far beyond sync producer's 384k #/s) So > I doubt if it is the case of maxing out network or procuder. > > === > and,I post the thread from google group: > https://groups.google.com/forum/?fromgroups#!forum/kafka-dev > > On Wed, Mar 28, 2012 at 3:25 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > >> Hi, >> >> How many producers are you using and on how many machines? Also what is >> the >> network speed (1gbit?) Likely you are maxing out either the producer >> machine(s) or the network itself. >> >> Also, quick question, how did you come across this address? This address >> has been dead for a while now, and the new mailing list is on Apache so I >> just want to make sure we don't have a dangling pointer somewhere. Would >> be >> good to move the discussion to the Apache list so others can find it. >> >> -Jay >> >> On Tue, Mar 27, 2012 at 2:10 AM, mmLiu <diveintotomor...@gmail.com> >> wrote: >> >> > I tested the performance of producer with >> ./bin/kafka-producer-perf-test.sh >> > >> > when 1 broker: >> > 20.3164 M/s,213033.3823#/s (async) >> > >> > when 2 brokers: >> > 37.4754 M/s,392958.1892#/s (async) >> > >> > when 3 brokers(perf stops increase) >> > 36.6925 M/s, 384748.5668#/s (async) >> > >> > when 4 brokers(even decrease): >> > 33.2244 M/s, 348383.5006 #/s (async) >> > >> > >> > Any idea why? >> > >> > FYI:the performance of sync producer increases(almost) linearly as >> number >> > of brokers increases. >> > > > > -- > Best Regards > > ---------------------- > 刘明敏 | mmLiu > > >
