Re: Strategies for improving Consumer throughput

2013-10-02 Thread Graeme Wallace
Ok. - so we figured out what the problem was with the consumers lagging behind. We were pushing 800Mbits/sec+ to the consumer interface - so the 1Gb network interface was maxed out. Graeme On Wed, Oct 2, 2013 at 2:35 PM, Graeme Wallace < graeme.wall...@farecompare.com> wrote: > Yes, definit

RE: Strategies for improving Consumer throughput

2013-10-02 Thread Yu, Libo
, October 02, 2013 4:36 PM To: users Subject: Re: Strategies for improving Consumer throughput Yes, definitely consumers are behind - we can see from examining the offsets On Wed, Oct 2, 2013 at 1:59 PM, Joe Stein wrote: > Are you sure the consumers are behind? could the pause be because

Re: Strategies for improving Consumer throughput

2013-10-02 Thread Graeme Wallace
Yes, definitely consumers are behind - we can see from examining the offsets On Wed, Oct 2, 2013 at 1:59 PM, Joe Stein wrote: > Are you sure the consumers are behind? could the pause be because the > stream is empty and producing messages is what is behind the consumption? > > What if you shut

Re: Strategies for improving Consumer throughput

2013-10-02 Thread Graeme Wallace
This is with 0.8 On Wed, Oct 2, 2013 at 2:00 PM, Philip O'Toole wrote: > Is this with 0.7 or 0.8? > > > On Wed, Oct 2, 2013 at 12:59 PM, Joe Stein wrote: > > > Are you sure the consumers are behind? could the pause be because the > > stream is empty and producing messages is what is behind the

Re: Strategies for improving Consumer throughput

2013-10-02 Thread Philip O'Toole
Is this with 0.7 or 0.8? On Wed, Oct 2, 2013 at 12:59 PM, Joe Stein wrote: > Are you sure the consumers are behind? could the pause be because the > stream is empty and producing messages is what is behind the consumption? > > What if you shut off your consumers for 5 minutes and then start the

Re: Strategies for improving Consumer throughput

2013-10-02 Thread Joe Stein
Are you sure the consumers are behind? could the pause be because the stream is empty and producing messages is what is behind the consumption? What if you shut off your consumers for 5 minutes and then start them again do the consumers behave the same way? /**