On 06/29/2011 10:37 AM, Marnie McCormack wrote:
Some useful reading links, for the concept in JMS terms/other
implementations (which users might expect us to support/be close ot
semantically). I'm not saying any of these approaches are exactly correct,
but that the logical behaviour they support is similar.
http://activemq.apache.org/message-groups.html
http://www.ibm.com/developerworks/websphere/library/techarticles/0602_currie/0602_currie.html
http://forum.springsource.org/showthread.php?48798-Spring-2.5.1-JMS-1.1-and-distributed-jobs-implementation
The activemq document gives a good clear description. They are implementing
Gordon's mode 2: stick to a consumer for as long as it lives.
They also raise an interesting point:
"If you have existing messages in the broker and add consumers at a later stage,
it is a good idea to delay message dispatch start until all consumers are
present (or at least to give enough time for them to subscribe). If you don't do
that the first consumer will probably acquire all message groups and all
messages will be dispatched to it. You can achieve this by using
consumersBeforeDispatchStarts and timeBeforeDispatchStarts destination policies. "
In other words you would like to be able to shift load from a busy consumer to a
less busy one, or have a random distribution of load that will even out on average.
Gordon's mode 1 (stick to a consumer only as long as it has unacked messages)
doesn't really do this: the only time it allows load to shift is when the queue
is idle long enough for the consumer to ack all its pre-fetched messages. But if
the queue is idle then we're not under heavy load which is where load balancing
is most important.
This makes me think there is a case for a 3rd mode of stickiness: non-stick
mode. In this mode we randomly chose the next consumer when the current consumer
acks. Clearly this will lower the possible msg/sec throughput since you are
effectively doing synchronous messaging. However it may still be relevant where
the msg/sec throughput is low anyway but the need for fair load balancing is
high - e.g. very large messages or messages that require a long time to process.
In that case the extra time added by synchronous messages may be negligible, but
the automatic distribution of load is very valuable.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
