Thanks Gordon. I'll fold your example into our code. The only reaon we have a queue per exchange is that we can monitor queue lengths for individual instruments in the event of delays on our pricing. If it is optimal to have one queue per session then we can implement that.
Thanks for your help. Regards, Dave -----Original Message----- From: Gordon Sim [mailto:[email protected]] Sent: 26 February 2010 13:57 To: David Stewart Cc: [email protected]; [email protected] Subject: Re: SubscriptionManager performance problem. On 02/26/2010 10:13 AM, David Stewart wrote: > The session was synchronous. Using the AsyncSession brought the 75 seconds > down to 5-10 which is a fantastic improvement. > The bottleneck still appears to be the SessionManager though. SessionManager::subscribe() is synchronous, which will impact the rate. One tip is to turn off message flow when you subscribe (use SubscriptionSettings(FlowControl::messageWindow(0))) and then add credit to resize the window on each subscription after you are set up (Subscription::grantMessageCredit() or Session::messageFlow()) to enable messages to flow (this can be asynchronous). I get through 20000 in ~25 secs on my laptop with that (see attached). However can I ask why you need a separate queue per exchange? Could you instead have one queue bound into each of the exchanges? That would be far more efficient (both for setup and at runtime) and would still give you all the messages sent to any of those exchanges. > > I should mention that we're running a vc90 C++ client against a vc90 C++ > broker. Could the broker be the problem? > Should I see better performance from a linux broker? > > -----Original Message----- > From: Gordon Sim [mailto:[email protected]] > Sent: 25 February 2010 17:39 > To: [email protected] > Cc: David Stewart; [email protected] > Subject: Re: SubscriptionManager performance problem. > > On 02/25/2010 04:51 PM, David Stewart wrote: >> Hi all, >> we are running a bridge between our old middleware and qpid system which at >> startup queries the existing middleware for the number of broadcast groups >> it knows about. It is a pricing system so there are ~20000. >> >> The bridge creates a fanout exchange for each broadcast group, creates a >> queue and binds it to the exchange. All this takes ~75 seconds for 20000. >> Not a problem. >> >> When we add a SubscriptionManager.subscribe() to the loop we get though 1000 >> requests in ~3 minutes. >> I have created an example below which exhibits the problem. The question are >> we using the SubscriptionManager incorrectly? > > Is oSession an AsyncSession or a SyncSession (a plain Session is a > SyncSession)? Each call to SubscriptionManager::subscribe() will involve a > sync() which will certainly reduce the rate you can get through them, but > 1000 in ~3 minutes seems slow even then. > >> Is there a better way for us to achieve the result we require? >> >> for (int i=0; i< 20000; ++i) { >> std::stringstream ss; ss<< "listener"<< i; >> >> >> // Try and declare the exchange. Will succeed even if it already >> exists. >> oSession.exchangeDeclare(qpid::client::arg::exchange=ss.str(), >> qpid::client::arg::type="fanout", >> >> qpid::client::arg::alternateExchange=std::string(), >> >> qpid::client::arg::passive=false, >> >> qpid::client::arg::durable=true); >> >> oSession.queueDeclare(qpid::client::arg::queue=ss.str(), >> qpid::client::arg::exclusive=true, >> >> qpid::client::arg::autoDelete=false); >> >> oSession.exchangeBind(qpid::client::arg::exchange=ss.str(), >> >> qpid::client::arg::queue=ss.str(), >> >> qpid::client::arg::bindingKey=ss.str()); >> >> oSubscriptionManager.subscribe(*this, ss.str()); } >> >> Regards, >> Dave >> >> The information contained in this email is strictly confidential and for the >> use of the addressee only, unless otherwise indicated. If you are not the >> intended recipient, please do not read, copy, use or disclose to others this >> message or any attachment. Please also notify the sender by replying to this >> email or by telephone (+44 (0)20 7896 0011) and then delete the email and >> any copies of it. Opinions, conclusions (etc.) that do not relate to the >> official business of this company shall be understood as neither given nor >> endorsed by it. IG Index Ltd is a company registered in England and Wales >> under number 01190902. VAT registration number 761 2978 07. Registered >> Office: Friars House, 157-168 Blackfriars Road, London SE1 8EZ. Authorised >> and regulated by the Financial Services Authority. FSA Register number >> 114059. >> >> --------------------------------------------------------------------- >> Apache Qpid - AMQP Messaging Implementation >> Project: http://qpid.apache.org >> Use/Interact: mailto:[email protected] >> > > > The information contained in this email is strictly confidential and for the > use of the addressee only, unless otherwise indicated. If you are not the > intended recipient, please do not read, copy, use or disclose to others this > message or any attachment. Please also notify the sender by replying to this > email or by telephone (+44 (0)20 7896 0011) and then delete the email and any > copies of it. Opinions, conclusions (etc.) that do not relate to the official > business of this company shall be understood as neither given nor endorsed by > it. IG Index Ltd is a company registered in England and Wales under number > 01190902. VAT registration number 761 2978 07. Registered Office: Friars > House, 157-168 Blackfriars Road, London SE1 8EZ. Authorised and regulated by > the Financial Services Authority. FSA Register number 114059. The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete the email and any copies of it. Opinions, conclusions (etc.) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG Index Ltd is a company registered in England and Wales under number 01190902. VAT registration number 761 2978 07. Registered Office: Friars House, 157-168 Blackfriars Road, London SE1 8EZ. Authorised and regulated by the Financial Services Authority. FSA Register number 114059. --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
