Re: Flow Control/Performance tuning help required

2011-09-28 Thread Gordon Sim
On 09/27/2011 07:29 PM, Fraser Adams wrote: Hi Gordon, This is slightly concerning. This seems a bit of a catch 22: for optimum performance it seems best to acknowledge fairly infrequently. It is a balance. If you leave it too infrequent then you have a much larger set of unacknowledged

Re: Flow Control/Performance tuning help required

2011-09-28 Thread Gordon Sim
On 09/28/2011 07:47 AM, Gordon Sim wrote: On 09/27/2011 07:29 PM, Fraser Adams wrote: Re A further question is whether you need acknowledgements at all? surely if I don't acknowledge at all then the messages just remain on the broker and it potentially attempts to resend them. Certainly if I

Re: Flow Control/Performance tuning help required

2011-09-27 Thread Gordon Sim
On 09/25/2011 07:43 PM, Fraser Adams wrote: This is really freaky why does the consumer performance drop off dramatically when the ring queue is full. Is it a flow control thing? No it is not a flow control related issue (that is controlled through the receivers capacity by the way). When a

Re: Flow Control/Performance tuning help required

2011-09-27 Thread Fraser Adams
Hi Gordon, This is slightly concerning. This seems a bit of a catch 22: for optimum performance it seems best to acknowledge fairly infrequently. Re A further question is whether you need acknowledgements at all? surely if I don't acknowledge at all then the messages just remain on the

Re: Flow Control/Performance tuning help required

2011-09-25 Thread Fraser Adams
Hi Andy/Gordon et al. I really could do with some help from performance gurus. OK So I think I've reproduced some of the symptoms I described in my earlier email. I used the attached demo producer/consumer qpid::messaging clients. So if I run ./ItemConsumer that creates my perftest queue

Flow Control/Performance tuning help required

2011-09-23 Thread Fraser Adams
Hello all, I was chatting to some colleagues yesterday who are trying to do some stress testing and have noticed some weird results. I'm afraid I've not personally reproduced this yet, but I wanted to post on a Friday whilst the list was more active. The set up is firing off messages of

Re: Flow Control/Performance tuning help required

2011-09-23 Thread Andy Goldstein
Hi Fraser, How many messages can the ring queue hold before it starts dropping old messages to make room for new ones? Andy On Sep 23, 2011, at 5:21 AM, Fraser Adams wrote: Hello all, I was chatting to some colleagues yesterday who are trying to do some stress testing and have noticed

Re: Flow Control/Performance tuning help required

2011-09-23 Thread Fraser Adams
Hi Andy, I'm afraid that I can't tell you for sure as I'm doing this a bit by remote control (I've tasked some of my developers to try and replicate the MRG whitepaper throughput results to give us a baseline top level performance figure). However when I last spoke to them they had tried

Re: Flow Control/Performance tuning help required

2011-09-23 Thread Andy Goldstein
As an experiment, try lowering the # of worker threads for the broker. For example, we saw an order of magnitude increase in performance when we dropped worker threads from 8 to 2 (on a 48-core server). Our test involved creating a ring queue with a max queue count of 250,000 messages. We

Re: Flow Control/Performance tuning help required

2011-09-23 Thread Fraser Adams
I'll mention that to the guys when I get back to the office. Though it seems a bit counterintuitive to me I'd have thought that having a lower number of worker threads wouldn't utilise the available cores. By logic running two (or even eight) worker threads on your 48 core server seems low -

Re: Flow Control/Performance tuning help required

2011-09-23 Thread Andy Goldstein
On Sep 23, 2011, at 9:11 AM, Fraser Adams wrote: I'll mention that to the guys when I get back to the office. Though it seems a bit counterintuitive to me I'd have thought that having a lower number of worker threads wouldn't utilise the available cores. By logic running two (or even