One order of magnitude of difference sounds about right, though, no? I don't know. I'm by no means a Kafka expert and we don't know much about the other variables of your situation, so it's hard to tell (at least for me) if there could be another explanation...
-- Felix On Tue, Jan 10, 2012 at 5:30 PM, Pierre-Yves Ritschard <p...@spootnik.org>wrote: > Surely, > > I just didn't expect such dramatically low numbers > > On Tue, Jan 10, 2012 at 11:27 PM, Felix GV <fe...@mate1inc.com> wrote: > > Maybe I'm overlooking something, but the first thing that came to my mind > > is: wouldn't you get no network latency at all on your local box if > > everything runs on the same machine? > > > > On EC2, the network latency would bring your overal throughput down, > > especially with a sync producer, wouldn't it? > > > > -- > > Felix > > > > > > > > On Tue, Jan 10, 2012 at 5:19 PM, Pierre-Yves Ritschard <p...@spootnik.org > >wrote: > > > >> Hi, > >> > >> One of my cluster runs on ec2 and I noticed that I have rather slow > >> throughput, less than 500 per second. I isolated the trouble on the > >> following setup > >> > >> 1 producer instance (m1.large) > >> 5 consumer instances also running a broker (m2.xlarge) > >> 1 zookeeper instance (m1.large) > >> > >> I am using a sync producer and have 60 partitions. > >> > >> On my local box, with everything running I would several thousand per > >> second. > >> >
