Hello group,
we were using ActiveMQ Artemis and it's predecessor HornetQ over years
now but never came to the point where we had to use clustering and ha.
Now this has changed and I have worked through the examples to setup a
ha cluster with 6 nodes (3 live, 3 backup).
While this works like a charm I have some major problems in the way we
use the messaging I want to explain in the following.
We have some sort of network devices attached to our software and
changes to the configuration causes "orders" to be placed into an order
queue (only one global queue). While processing these orders, messages
are produced for devices and placed into an outbox queue (one per
device). Our requirement to this order queue is sequential processing
(the first message from a queue for a device has to be completely
processed before the next one is picked). So far we managed this using
the concept of grouping messages and filtering via a certain message
property (i.e. the device id). This offered the possibility to handle
this inside our software because the broker dispatches all messages with
the same group to the same consumer with the matching filter, which is
great.
Moreover we have a so called event topic where we distribute messages
between the parts of the software which is made up of several
independent processes. This is done through a classic topic (one address
with several queues attached).
Node A Node B Node V
___________ ___________ ___________
|___QUEUE___| ||||QUEUE|||| |___QUEUE___|
| ^ |
| | |
| | |
Consumer 1 Producer Consumer 2
With clustering a big problem arises from these requirements (I tried to
picture that above):
For of the queues it is possible that the producer is connected to
node B and consumers connected to node A and C leading to B building up
a queue with a large message count while the two consumers starve.
I already worked through the concepts of message redistribution, diverts
and bridges but none of them seems to solve this problem. Message
redistribution only kicks in, when there are no more consumers on a
queue of a distinct node. But for our order queue there might well be
several consumers on different nodes with different filters, so its
quite difficult to say which message should be dispatched to which node.
Diverting could do more on this but I do not see how this could work
with the concept of failover and failback, because we could never know
in advance to which of the available node a consumer would connect after
failover and its messages could well be all on a failed node.
We would like to see a queue as an abstract source of messages for the
consumers they can read from never mind to which node of the cluster
they are connected. I understand that this would mean that message
reception and removal would have to be replicated to all members of a
cluster (event the backup nodes) which would cause high load I guess. I
am hoping that someone has a hint for me how we could address this.
Kind regards
Sebastian
