RE: [JBoss-dev] jbossmq & load balancing across vms
Basically because of efficiency. Because the possibility exists that when a receive is done the receiver may already be in the list of receivers (due to receives with timeouts) we have to check that a receiver is not already in the list before adding them to the list. This search O(n) for a LinkedList and constant time for a HashSet. If you are absolutely desparate for the round-robin allocation of messages then I would re-implement keeping both the LinkedList and the HashSet. Use the HashSet to check if the receiver is already receiving and the LinkedList to maintain the order of the requests. The code I am talking about is the org.jboss.mq.server.BasicQueue class. Cheers, David. > -Original Message- > From: Jason Dillon [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, August 22, 2001 1:50 PM > To: '[EMAIL PROTECTED]' > Subject: RE: [JBoss-dev] jbossmq & load balancing across vms > > > > What this means is that if you have multiple receivers > waiting for a message > > from a queue and a message arrives then it will be > immediately delivered to > > receiver A. Receiver A is now removed from the HashSet, > however if receiver > > A finishes its processing and requests another message > before another > > message is sent to the queue it will be added back to the > HashSet. When the > > next message arrives set will again be sent to receiver A > because in the > > HashSets ordering receiver A is clearly ahead of receiver B. > > Why a HashSet and not a LinkedList? > > --jason > > > ___ > Jboss-development mailing list > [EMAIL PROTECTED] > http://lists.sourceforge.net/lists/listinfo/jboss-development > ___ Jboss-development mailing list [EMAIL PROTECTED] http://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq & load balancing across vms
> What this means is that if you have multiple receivers waiting for a message > from a queue and a message arrives then it will be immediately delivered to > receiver A. Receiver A is now removed from the HashSet, however if receiver > A finishes its processing and requests another message before another > message is sent to the queue it will be added back to the HashSet. When the > next message arrives set will again be sent to receiver A because in the > HashSets ordering receiver A is clearly ahead of receiver B. Why a HashSet and not a LinkedList? --jason ___ Jboss-development mailing list [EMAIL PROTECTED] http://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq & load balancing across vms
I'm not up-to-date on javaspaces but with jms you can certainly have multiple receivers on the same message queue, you can also have multiple senders. The JMS spec does not specify any policy for how messages are distributed between different receivers though. As for message order, it is only guaranteed amongst messages sent to the same queue from the same sender, that is if you receive two messages from the same queue sent by the same sender they will arrive in the order sent (except for some cases involving different priorities and the like). Cheers, David > -Original Message- > From: David Jencks [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, August 22, 2001 12:48 PM > To: [EMAIL PROTECTED] > Subject: Re: [JBoss-dev] jbossmq & load balancing across vms > > > I am not an expert Is this sharing of a message queue > receiving end > really spec compliant? How does it relate to > order-of-messages guarantees? > It seems to me this may be the conceptual difference between > message queues > and javaspaces..?? > > david jencks > > On 2001.08.21 19:06:08 -0400 Jason Dillon wrote: > > Hello again, I am running into a problem with the latest JBossMQ wrt > > using > > it to load balance across machines. > > > > I have one machine which acts as a message router, running > JBossMQ. All > > other machines use it for their JMSProvider. When I have > more than one > > MDB > > attached to a single queue, and a user-thread adding > messages over and > > over, > > it looks like only one of the MDB's will ever read any messages. > > > > I do not remember seeing this before. It looks like when I > startup a new > > MDB that 50% of the time it will take over and the others > will remain > > idle, > > or it will simply never read any new messages. > > > > I have each of these MDB setup with: > > > > 1 > > 1 > > > > Which could be causing problem... but honestly I don't > really know what > > these two setting actually do. I wanted to have these > deployed MDB's to > > only process one message at a time, which this seems to > accomplish most > > of > > the time. > > > > Any ways, I am concerended about how JBossMQ is pushing > messages over to > > consumers. I mentioned before that if one producer added > 1k messages to > > a > > queue, then one MDB was attached which would suck up all of > the messages. > > When a second MDB was attached it would not have anything > todo. Hiram > > mentioned something about this, but I am not really sure > what the plans > > were > > to fix this. > > > > I would like to get this fixed soon, as I am fairly > dependent on this > > behavior. > > > > From my perspective, there are two major issues: > > > > 1) JBossMQ does not seem to trade off between consumers > (ie. round-robin > > or > > whatever). > > > > 2) JBossMQ does not have any configurable limits on how > messages are > > pushed > > from the server to clients/consumers. > > > > I understand why JBossMQ wants to push over bulk messages > to consumers, > > but > > that is only really useful in some applications. In my > case, I would > > rather > > that it pushed over smallers chunks (or even 1 msg at a > time) to clients > > so > > that I could utilize the cluster of machines that I have to > distribute > > the > > processing load. > > > > So, to the JBossMQ folks, does this sound like it would be easy to > > implement? Do you have any suggestions on how it might be > done (quick > > fix > > and full blown god JMS provider version)? I am willing to > look into this > > more, but I could use some pointers on which bits need to > be tweaked. > > > > Any ideas? > > > > --jason > > > > > > ___ > > Jboss-development mailing list > > [EMAIL PROTECTED] > > http://lists.sourceforge.net/lists/listinfo/jboss-development > > > > > > ___ > Jboss-development mailing list > [EMAIL PROTECTED] > http://lists.sourceforge.net/lists/listinfo/jboss-development > ___ Jboss-development mailing list [EMAIL PROTECTED] http://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq & load balancing across vms
It does not say... this is one of the grey-areas of the spec. Most providers will round-robin over multipule recievers. It is really too bad that this is not more concreate, as JMS would provide a nice mech. for distributing over a large group of machines, and allow for ordering and such. I think that I will probably drop JMS for this as soon as I have time to rearch. under a different distribution system (like JavaSpaces). JMS is too ugly and too vendor specific. I will probably still keep it around for async events from beans/to beans though. Just not as the backend for distributing load. --jason On Tue, 21 Aug 2001, David Jencks wrote: > I am not an expert Is this sharing of a message queue receiving end > really spec compliant? How does it relate to order-of-messages guarantees? > It seems to me this may be the conceptual difference between message queues > and javaspaces..?? > > david jencks > > On 2001.08.21 19:06:08 -0400 Jason Dillon wrote: > > Hello again, I am running into a problem with the latest JBossMQ wrt > > using > > it to load balance across machines. > > > > I have one machine which acts as a message router, running JBossMQ. All > > other machines use it for their JMSProvider. When I have more than one > > MDB > > attached to a single queue, and a user-thread adding messages over and > > over, > > it looks like only one of the MDB's will ever read any messages. > > > > I do not remember seeing this before. It looks like when I startup a new > > MDB that 50% of the time it will take over and the others will remain > > idle, > > or it will simply never read any new messages. > > > > I have each of these MDB setup with: > > > > 1 > > 1 > > > > Which could be causing problem... but honestly I don't really know what > > these two setting actually do. I wanted to have these deployed MDB's to > > only process one message at a time, which this seems to accomplish most > > of > > the time. > > > > Any ways, I am concerended about how JBossMQ is pushing messages over to > > consumers. I mentioned before that if one producer added 1k messages to > > a > > queue, then one MDB was attached which would suck up all of the messages. > > When a second MDB was attached it would not have anything todo. Hiram > > mentioned something about this, but I am not really sure what the plans > > were > > to fix this. > > > > I would like to get this fixed soon, as I am fairly dependent on this > > behavior. > > > > From my perspective, there are two major issues: > > > > 1) JBossMQ does not seem to trade off between consumers (ie. round-robin > > or > > whatever). > > > > 2) JBossMQ does not have any configurable limits on how messages are > > pushed > > from the server to clients/consumers. > > > > I understand why JBossMQ wants to push over bulk messages to consumers, > > but > > that is only really useful in some applications. In my case, I would > > rather > > that it pushed over smallers chunks (or even 1 msg at a time) to clients > > so > > that I could utilize the cluster of machines that I have to distribute > > the > > processing load. > > > > So, to the JBossMQ folks, does this sound like it would be easy to > > implement? Do you have any suggestions on how it might be done (quick > > fix > > and full blown god JMS provider version)? I am willing to look into this > > more, but I could use some pointers on which bits need to be tweaked. > > > > Any ideas? > > > > --jason > > > > > > ___ > > Jboss-development mailing list > > [EMAIL PROTECTED] > > http://lists.sourceforge.net/lists/listinfo/jboss-development > > > > > > ___ > Jboss-development mailing list > [EMAIL PROTECTED] > http://lists.sourceforge.net/lists/listinfo/jboss-development > ___ Jboss-development mailing list [EMAIL PROTECTED] http://lists.sourceforge.net/lists/listinfo/jboss-development
Re: [JBoss-dev] jbossmq & load balancing across vms
I am not an expert Is this sharing of a message queue receiving end really spec compliant? How does it relate to order-of-messages guarantees? It seems to me this may be the conceptual difference between message queues and javaspaces..?? david jencks On 2001.08.21 19:06:08 -0400 Jason Dillon wrote: > Hello again, I am running into a problem with the latest JBossMQ wrt > using > it to load balance across machines. > > I have one machine which acts as a message router, running JBossMQ. All > other machines use it for their JMSProvider. When I have more than one > MDB > attached to a single queue, and a user-thread adding messages over and > over, > it looks like only one of the MDB's will ever read any messages. > > I do not remember seeing this before. It looks like when I startup a new > MDB that 50% of the time it will take over and the others will remain > idle, > or it will simply never read any new messages. > > I have each of these MDB setup with: > > 1 > 1 > > Which could be causing problem... but honestly I don't really know what > these two setting actually do. I wanted to have these deployed MDB's to > only process one message at a time, which this seems to accomplish most > of > the time. > > Any ways, I am concerended about how JBossMQ is pushing messages over to > consumers. I mentioned before that if one producer added 1k messages to > a > queue, then one MDB was attached which would suck up all of the messages. > When a second MDB was attached it would not have anything todo. Hiram > mentioned something about this, but I am not really sure what the plans > were > to fix this. > > I would like to get this fixed soon, as I am fairly dependent on this > behavior. > > From my perspective, there are two major issues: > > 1) JBossMQ does not seem to trade off between consumers (ie. round-robin > or > whatever). > > 2) JBossMQ does not have any configurable limits on how messages are > pushed > from the server to clients/consumers. > > I understand why JBossMQ wants to push over bulk messages to consumers, > but > that is only really useful in some applications. In my case, I would > rather > that it pushed over smallers chunks (or even 1 msg at a time) to clients > so > that I could utilize the cluster of machines that I have to distribute > the > processing load. > > So, to the JBossMQ folks, does this sound like it would be easy to > implement? Do you have any suggestions on how it might be done (quick > fix > and full blown god JMS provider version)? I am willing to look into this > more, but I could use some pointers on which bits need to be tweaked. > > Any ideas? > > --jason > > > ___ > Jboss-development mailing list > [EMAIL PROTECTED] > http://lists.sourceforge.net/lists/listinfo/jboss-development > > ___ Jboss-development mailing list [EMAIL PROTECTED] http://lists.sourceforge.net/lists/listinfo/jboss-development
RE: [JBoss-dev] jbossmq & load balancing across vms
Hi Jason, The way we implemented the message delivery this behaviour could happen, but only when messages are arriving more slowly than they are being requested. This is because the set of receivers waiting for messages is a HashSet so that if a receiver sends a request for a message more than once (which happens when using a receive with a timeout) it won't get added more than once. What this means is that if you have multiple receivers waiting for a message from a queue and a message arrives then it will be immediately delivered to receiver A. Receiver A is now removed from the HashSet, however if receiver A finishes its processing and requests another message before another message is sent to the queue it will be added back to the HashSet. When the next message arrives set will again be sent to receiver A because in the HashSets ordering receiver A is clearly ahead of receiver B. We didn't see any problem with this at the time, under heavy client processing loads where messages are waiting on the queue as requests are made then the messages will be shared out amongst the clients. When there is light loads then the messages will tend to be delivered to the same client. We could implement an ordering on the server-side but I still don't know if it is required. Remember this only happens when your clients are going fast enough that they actually have to wait for messages to arrive. If messages are being produced faster than the clients can process them then they will be shared out amongst all the clients. Cheers David > -Original Message- > From: Jason Dillon [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, August 22, 2001 11:06 AM > To: [EMAIL PROTECTED] > Subject: [JBoss-dev] jbossmq & load balancing across vms > > > Hello again, I am running into a problem with the latest > JBossMQ wrt using > it to load balance across machines. > > I have one machine which acts as a message router, running > JBossMQ. All > other machines use it for their JMSProvider. When I have > more than one MDB > attached to a single queue, and a user-thread adding messages > over and over, > it looks like only one of the MDB's will ever read any messages. > > I do not remember seeing this before. It looks like when I > startup a new > MDB that 50% of the time it will take over and the others > will remain idle, > or it will simply never read any new messages. > > I have each of these MDB setup with: > > 1 > 1 > > Which could be causing problem... but honestly I don't really > know what > these two setting actually do. I wanted to have these > deployed MDB's to > only process one message at a time, which this seems to > accomplish most of > the time. > > Any ways, I am concerended about how JBossMQ is pushing > messages over to > consumers. I mentioned before that if one producer added 1k > messages to a > queue, then one MDB was attached which would suck up all of > the messages. > When a second MDB was attached it would not have anything todo. Hiram > mentioned something about this, but I am not really sure what > the plans were > to fix this. > > I would like to get this fixed soon, as I am fairly dependent on this > behavior. > > From my perspective, there are two major issues: > > 1) JBossMQ does not seem to trade off between consumers (ie. > round-robin or > whatever). > > 2) JBossMQ does not have any configurable limits on how > messages are pushed > from the server to clients/consumers. > > I understand why JBossMQ wants to push over bulk messages to > consumers, but > that is only really useful in some applications. In my case, > I would rather > that it pushed over smallers chunks (or even 1 msg at a time) > to clients so > that I could utilize the cluster of machines that I have to > distribute the > processing load. > > So, to the JBossMQ folks, does this sound like it would be easy to > implement? Do you have any suggestions on how it might be > done (quick fix > and full blown god JMS provider version)? I am willing to > look into this > more, but I could use some pointers on which bits need to be tweaked. > > Any ideas? > > --jason > > > ___ > Jboss-development mailing list > [EMAIL PROTECTED] > http://lists.sourceforge.net/lists/listinfo/jboss-development > ___ Jboss-development mailing list [EMAIL PROTECTED] http://lists.sourceforge.net/lists/listinfo/jboss-development
