Yup, I suspect the client library. On Thu, Sep 15, 2016 at 10:28 AM, kant kodali <kanth...@gmail.com> wrote:
> Hi Ben, > I can give that a try but can you tell me the suspicion or motivation > behind it? > other words you think single partition and single broker should be > comparable to > the setup I had with NATS and NSQ except you suspect the client library or > something? > Thanks,Kant > > > > > > > On Thu, Sep 15, 2016 2:16 AM, Ben Davison ben.davi...@7digital.com > wrote: > > Hi Kant, > > > > > I was following the other thread, can you try using a different > > benchmarking client for a test. > > > > > https://grey-boundary.io/load-testing-apache-kafka-on-aws/ > > > > > Ben > > > > > On Thursday, 15 September 2016, kant kodali <kanth...@gmail.com> wrote: > > > > > with Kafka I tried it with 10 messages with single broker and only one >> > > partiton >> > > that looked instantaneous and ~5K messages/sec for the data size of 1KB >> > > I tried it with 1000 messages that looked instantaneous as well ~5K >> > > messages/sec >> > > for the data size of 1KBI tried it with 10K messages with single broker >> > > and only >> > > one partiton things started to go down ~1K messages/sec for the data size >> > > of >> > > 1KB >> > > having only one partition on a single broker is a bad? My goal is to run >> > > some >> > > basic benchmarks on NATS & NSQ & KAFKA >> > > I have the same environment for all three (NATS & NSQ & KAFKA) >> > > a broker on Machine 1producer on Machine 2Consumer on Machine 3 >> > > with a data size of 1KB (so each message is 1KB ) and m4.xlarge aws >> > > instance. >> > > I have pushed 300K messages with NATS and it was able to handle easily and >> > > receive throughput was 5K messages/secI have pushed 300K messages and NSQ >> > > and >> > > receive throughput was 2K messages/secI am unable to push 300K messages >> > > with >> > > Kafka with the above configuration and environmentso at this point my >> > > biggest >> > > question is what is the fair setup for Kafka so its comparable with NATS >> > > and >> > > NSQ? >> > > kant >> > > > > > -- > > > > > > > > This email, including attachments, is private and confidential. If you > have > received this email in error please notify the sender and delete it from > your system. Emails are not secure and may contain viruses. No liability > can be accepted for viruses that might be transferred by this email or any > attachment. Any unauthorised copying of this message or unauthorised > distribution and publication of the information contained herein are > prohibited. > > > > > 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB. > > Registered in England and Wales. Registered No. 04843573. -- This email, including attachments, is private and confidential. If you have received this email in error please notify the sender and delete it from your system. Emails are not secure and may contain viruses. No liability can be accepted for viruses that might be transferred by this email or any attachment. Any unauthorised copying of this message or unauthorised distribution and publication of the information contained herein are prohibited. 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB. Registered in England and Wales. Registered No. 04843573.
