On 06/28/2011 06:39 AM, Gordon Sim wrote:
On 06/28/2011 09:57 AM, Marnie McCormack wrote:
In JMS, the sequence id would be used to maintain order, but the expectation
is that the group are processed as a whole on receipt of the final message
in the group rather than simply shared out between consumers and processed
in order.
I think that is a third variation (further restriction really) on the use case.
I.e. where message group is used to piece together a large unit from discrete
messages.
So I think the requirements are:
(1) message group must always be processed in order
(2) all messages in a group are processed by the same consumer
(3) the consumer will only complete processing having received the 'final'
message in the group
What happens if the consumer dies or cancel's mid-group in this case? Do we
- replay the group from the beginning to another consumer?
- drop the rest of the group?
- continue sending to a new consumer from where the first consumer left off?
- In which case do we provide a flag to let the new consumer know it's
starting mid-group?
Where (3) implies (2) and (2) implies (1).
The third requirement essentially rules out an implementation of the first
requirement that refuses to deliver the next message in the group until the
previous message is accepted.
The second requirement is important where there is a desire to contain all the
necessary state for processing a group of messages within a single consumer. I
agree that is important and should be supported. However I think it is not
always required and it should be possible to drop that if not needed as that
could lead to more adaptive load-balancing while meeting the first requirement.
So I think the configuration involved is:
(a) identification of the header that signifies a group within a given queue,
which implicitly enables requirement (1)
(b) an indication that 'stickiness' of group to consumer is required (beyond
simply meeting the first requirement)
I think we can avoid the need for any explicit configuration of requirement (3).
Refusing to deliver the next message in the group until the previous message is
accepted is in my view unnecessary to meet (1) and could be emulated more or
less by controlling prefetch.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
