ah, my bad... I expect to have 20 consumer threads per instance. Sorry, so that'd be 600 consumer threads in the environment.
On Wed, Jan 11, 2012 at 10:14 AM, Praveen M <[email protected]> wrote: > Great, that helps. > > Would you have a similar recommendation for the consumer side? > > In my environment, I expect about 200 consumer threads per instance. Total > (30 instance * 200 threads/per instance) - 6000 consumer threads > > I was thinking I'd just use one connection per instance and then spin off > 200 sessions/consumers. > > my onMessage() callback is expected to take atleast 1 second to process (I > don't see a reason why a message should be submitted for async processing > if it takes lesser than that to process) > , so I assumed I must be ok with one connection. > > A latency is totally acceptable before a message gets picked up. > Is there any recommendations towards connection management per > instance for the consumer side of things? I'd like to hear what you think > about it. If I'm right to assume 1 connection per > instance should be good? > > Also, > Since the consumer side would add more parallel connections to the broker, > I'm curious upon the limits I could push the broker. > Is there any tested limit in terms of the number of connections that the > broker could support? > We are looking at 180 concurrent connections to the broker if I say 1 > connection per instance for all the consumers and 5 connections per > instance for enqueues Should that be ok? > > Also, are there any cons for having a lot of connections to the broker > which I should be aware of? > > Thanks a lot, > Praveen > > > On Wed, Jan 11, 2012 at 9:52 AM, Robbie Gemmell > <[email protected]>wrote: > >> 150 connections should be no problem. >> >> I would expect any locking on the client to be of little significance >> to overall performance when using persistent messages, hitting the >> disk is going to dominate overall performance. The issue with using >> multiple Sessions on a Connection with persistent messages is that it >> the Sessions are ultimately multiplexed onto the same TCP connection >> and their data processed sequentially on arrival at the broker. If for >> example you were to do 5 concurrent sends on 5 Sessions on the same >> Connection, they will be processed onto the connection sequentially >> and thus into the store sequentially upon arrival at the broker, >> whereas if you do 5 concurrent sends for Sessions on individual >> Connections there is scope for them to be processed into the store >> concurrently and enable some/all of them to be write-combined and >> synced to disk at the same time, reducing the required synchronous >> disk IO to perform the operations and thus providing increased overall >> performance/scalability by effectively 'doing more with less' so to >> speak. >> >> Robbie >> >> On 11 January 2012 16:24, Praveen M <[email protected]> wrote: >> > Hi Robbie, >> > >> > Thanks for writing. I just had a few followup questions. Please see >> in-line. >> > >> > Thanks a lot for helping out with this, >> > Praveen >> > >> > On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell < >> [email protected]>wrote: >> > >> >> Hi Praveen, >> >> >> >> If you are predominantly going to use persistent messages, then I >> >> would actually look at your publishers using a single single session >> >> per connection for the most part. >> >> >> >> The broker is currently able to best extract performance scaling from >> >> the persistent stores (more so the BDBstore as per Robs recent >> >> testing...though the results of which are again slightly out of date >> >> now due to the upgrade to BDB5 earlier today) by performing actions on >> >> a per-connection basis, so it can extract parrallism over commits to >> >> the store. Using a lower number of connections with 5-10 sessions each >> >> could be noticably slower at peak when using 150-300 publishers. >> >> >> > >> > that totally makes sense. But now that I'd have approximately (30 >> instance >> > * 5 connections) = 150 >> > connections to the broker, I wouldn't be overwhelming the broker right? >> I'm >> > guessing it shouldn't be a problem, But just want to run it by you? >> > >> > I understand that having separate connections is significantly faster as >> > each >> > connection would operate on it's own socket. >> > Just out of curiosity I'd like to know, what are the performance >> > degradation >> > causes when using sessions sharing the same connection? I know for one >> that >> > the session has to hold on creation locks (for creation) and connection >> > failover locks on send. >> > Do you think this will be the cause of the major bottle necks or is >> there >> > something else too? >> > (I'm sure you know the code back to front to answer this better) :) >> > >> > >> > >> >> If you are sending to a huge variety of Destinations then the way you >> >> are using your Producers seems appropriate; creating producers isnt >> >> necessarily that heavyweight an operation, but not doing so is >> >> undeniably far more efficient. >> >> >> > >> > ah, yep. that makes sense. I think I will stick to creating producers on >> > connection pooling >> > and creating the session >> > >> > >> >> >> >> I have actually never personally used sync_publish, but given that >> >> both the underlying sync() call and session.commit() hit the broker >> >> synchronously I would guess their performance should be pretty similar >> >> for persistent messages with a batch size of 1, in which case I would >> >> probably just go with transactions for 'pure JMS' purposes. If you >> >> ever were using batches of >1 message that would seem to favor use of >> >> transactions due to lowering the number of synchronous round trips to >> >> the broker which would be required, increasing performance. >> >> >> >> yep, I totally agree with you on using transactions over the >> sync_publish >> > option >> > for pure JMS purposes. >> > >> > >> >> Robbie >> >> >> >> On 10 January 2012 17:48, Praveen M <[email protected]> wrote: >> >> > Hi Robbie, >> >> > >> >> > Thanks for writing. >> >> > >> >> > Here is some more context for my 2nd question. >> >> > >> >> > I'm expected to have about 30 client instances connecting to one >> broker. >> >> > About 90% of the messages to the broker are expected to be persistent >> >> > messages. >> >> > >> >> > I haven't decided whether to use transactions or use the option >> >> > sync_publish. The >> >> > guarantee that we'd expect is that the broker receives the message >> before >> >> > proceeding to >> >> > send the next message. I'd be happy to hear your recommendations >> around >> >> > this. >> >> > >> >> > From the code I read, it looked like sync_publish was working to that >> >> > effect. But one thing I >> >> > saw was the sync() call in the client held to a connection failover >> lock, >> >> > and I wasn't sure if the sync() call was blocking. >> >> > Anyways, that said do you have a recommendation between transacted vs >> >> > sync_publish in a multi client environment (things going in >> parallel). >> >> I'd >> >> > really like to hear what you think about this. >> >> > >> >> > There will not be any batch enqueues (a top of my head). Almost all >> our >> >> > traffic will be having messages >> >> > enqueued one at a time. >> >> > >> >> > The number of destinations we have could be about 4000-5000. But yes, >> >> there >> >> > is a strong likelihood of one >> >> > queue being more popular and all the traffic getting routed there. We >> >> > sometimes receive bursty enqueues of a certain type >> >> > and is fair to assume that 5 or more enqueues of the same type to the >> >> same >> >> > queue is possible to happen concurrently. >> >> > >> >> > We plan to have own framework on top of Qpid which will have the >> ability >> >> to >> >> > load balance between the destinations >> >> > if one queue goes over a certain threshold (some hand-picked >> number..not >> >> > decided on yet) and then route traffic to another destination. >> >> > >> >> > And the overall traffic expected is somewhere between 10 million - 20 >> >> > million messages a day maybe. >> >> > >> >> > Please do let me know if you'd like to hear any other information. >> >> > >> >> > Thank you, >> >> > Praveen >> >> > >> >> > >> >> > But yes, we'd be using one of the two (whichever would perform best >> with >> >> > multiple concurrent clients). >> >> > >> >> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell < >> >> [email protected]>wrote: >> >> > >> >> >> Hi Praveen, >> >> >> >> >> >> Using either JNDI or Session.createQueue should work fine, but the >> >> >> obvious limitation of the latter is that the (possibly >> >> >> provider-specific) configuration is then part of your code (unless >> you >> >> >> look the information up somewhere before using it) and cant be >> changed >> >> >> without modifying the code. >> >> >> >> >> >> To better answer the other question, it would be good to first know >> >> >> more specifics about the application and its messaging model, eg: >> >> >> How many instances of the application are you likely to be running? >> >> >> Will all your messages be persistent? >> >> >> Will you always use transactions (and if so will it always of batch >> >> >> size 1 like below)? >> >> >> You mention having maybe 5-10 concurrent publishers, but how many >> >> >> different Destinations are you likely to be sending to in total? >> >> >> Are any of the above Destinations likely to be used a lot more than >> the >> >> >> others? >> >> >> >> >> >> Robbie >> >> >> >> >> >> On 10 January 2012 00:27, Praveen M <[email protected]> >> wrote: >> >> >> > Hi, >> >> >> > >> >> >> > I'm writing a JMS Client and have a few best practices >> questions. >> >> >> > >> >> >> > 1) I understand that there are different ways ways to create a >> >> reference >> >> >> to >> >> >> > a destination to enqueue a message (using JMS createQueue, using >> JNDI >> >> >> > lookup). >> >> >> > What would be the recommended way to use? >> >> >> > >> >> >> > 2) I create a connection to the broker on my application startup, >> and >> >> >> > create a session pool. on every enqueue, i checkout a session >> from my >> >> >> pool >> >> >> > use it and return to pool on enqueue. >> >> >> > Does this sound reasonable? (code sample below) >> >> >> > >> >> >> > What would be the pattern that is typically used/recommended >> for >> >> >> > session management/connection management in the client side? >> >> >> > >> >> >> > Or is this an overkill to do? Is CreateSession() and >> >> CreateProducer() >> >> >> > really cheap that it is just done on each enqueue? >> >> >> > In my use case, I'd expect quite some concurrency (assume n >> (maybe >> >> >> > 5-10) threads doing enqueues at the same time) and I'd prefer it >> to >> >> not >> >> >> > lock/block and go through asap. >> >> >> > >> >> >> > I do remember from reading the qpid client code that there is a >> >> lock >> >> >> > held on the connection object whenever a new session is created on >> >> that >> >> >> > connection. I'm assuming it is one of the places where my enqueue >> can >> >> >> slow >> >> >> > down >> >> >> > when going in parallel if it has to create a session for every >> >> enqueue >> >> >> > of a message. There might be more such blocks, which I'm not >> aware of. >> >> >> > >> >> >> > Taking performance into mind, what would be the best design for >> >> >> > session/connection management? >> >> >> > >> >> >> > A snippet of what I have currently is below, >> >> >> > >> >> >> > // a thread safe queue with the available session producer >> pairs >> >> which >> >> >> > can be used to create a message and enqueue. >> >> >> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>> >> >> >> > availableSessionProducerPairs = new >> >> ConcurrentLinkedQueue<Pair<Session, >> >> >> > MessageProducer>> (); >> >> >> > >> >> >> > // initialize the sessionProducerPairs, create upto n pairs >> >> >> > initializePool(Connection qpidConnection) { >> >> >> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ >> ) { >> >> >> > Session session = qpidConnection.createSession(true, >> >> >> > Session.SESSION_TRANSACTED); >> >> >> > MessageProducer producer = >> session.createProducer(null); >> >> >> > availableSessionProducerPairs.add(Pair.newPair(session, >> >> >> > producer)); >> >> >> > } >> >> >> > } >> >> >> > >> >> >> > // on enqueue we checkout a session/producer pair use it for the >> >> enqueue >> >> >> > and then return it back to the session/produer pair pool. >> >> >> > enqueueMessage(String queueName, byte[] message) { >> >> >> > // remove the session producer pair >> >> >> > Pair<Session, MessageProducer> sessionProducerPair = >> >> >> > availableSessionProducerPairs.poll(); >> >> >> > if(sessionProducerPair == null) { >> >> >> > // gack here and return null. >> >> >> > } >> >> >> > Session session = sessionProducerPair.getFirst(); >> >> >> > MessageProducer producer = sessionProducerPair.getSecond(); >> >> >> > BytesMessage jmsMessage = session.createBytesMessage(); >> >> >> > jmsMessage.writeBytes(message); >> >> >> > Queue queueRef = queueNameToReferenceMap.get(queueName); >> >> >> > producer.send(queueRef, jmsMessage, >> DeliveryMode.PERSISTENT, 0, >> >> >> 0); >> >> >> > session.commit(); >> >> >> > availableSessionProducersPair.add() >> >> >> > } >> >> >> > >> >> >> > >> >> >> > >> >> >> > Thank you, >> >> >> > -Praveen >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> Apache Qpid - AMQP Messaging Implementation >> >> >> Project: http://qpid.apache.org >> >> >> Use/Interact: mailto:[email protected] >> >> >> >> >> >> >> >> > >> >> > >> >> > -- >> >> > -Praveen >> >> >> >> --------------------------------------------------------------------- >> >> Apache Qpid - AMQP Messaging Implementation >> >> Project: http://qpid.apache.org >> >> Use/Interact: mailto:[email protected] >> >> >> >> >> > >> > >> > -- >> > -Praveen >> >> --------------------------------------------------------------------- >> Apache Qpid - AMQP Messaging Implementation >> Project: http://qpid.apache.org >> Use/Interact: mailto:[email protected] >> >> > > > -- > -Praveen > -- -Praveen
