Hi,
I have little experience using messaging systems, so I would like to hear
peoples opinion about a topology I am thinking about using.
The purpose of the system is to allow front-end servers (producers) to
asynchronously send commands to multiple back-end servers (consumers), and
allow the front-ends to collect the responses using temporary queues.
I do not care about reliability. All messages will be non-persistent and
will have a short time-to-live. If a response does not arrive within a
certain timeout, the front-end will handle that by, for example,
displaying an error message.
I do care about failover for back-end servers, and load balancing the
requests over the back-end servers.
To satisfy these requirements, I want to embed a broker inside every
front-end and have the back-ends connect to the brokers of front-ends.
For example, assume the following components are used:
- 2 front-ends: X and Y
- 2 business-logic back-ends: BL1 and BL2
- 4 compute-intensive back-ends: CI1-CI4
- 1 non-critical back-end: NC
Both X and Y have 3 queues (I am thinking about using queues at this
moment, but a single queue with message filters might work better and/or
be more convenient. I just have not investigated the latter enough yet.
Advice is appreciated):
- business-logic: bl
- compute-intensive: ci
- non-critical: nc
Here is the wiring. The back-ends know to which front-ends they must
connect by explicit configuration. The back-ends will connect to the
front-ends using the failover protocol with a single URI, so connections
to front-ends will automatically be restored in case of problems. I will
start a thread in the back-end for each connection to a front-end, so
messages from multiple front-ends are consumed simultaneously:
- both BL1 and BL2 connect to X.bl and Y.bl
- load balancing: both BL1 and BL2 consume messages and only as fast as
they can process them
- failover: if either BL1 or BL2 fails, the other will still handle
messages of both front-ends
- CI1 and CI2 connect to X.ci, and CI3 and CI4 connect to Y.ci
- load balacing: all back-ends, CI1-CI4, consume messages and only as
fast as they can process them
- failover: if one of them fails, the front-end from which it consumed
messages will still have another back-end consuming messages
- NC connects to X.nc and Y.nc
- load balancing: not necessary, NC is fast enough
- failover: not applicable because the back-end is non-critical
If you need load balancing you automatically get failover, and vice versa.
So if load balancing would be needed in the case of NC, then you
automatically get failover even though that is not a requirement for that
back-end type.
The brokers are load balanced by given each front-end its own broker
inside the same JVM. There is no failover for brokers, because the
assumption is that if the broker has problems, then the whole front-end
will have problems, so the front-end won't be sending messages anyway.
For the compute-intensive back-ends I chose to have the back-ends connect
to a single front-end to reduce the number of incoming connections of
brokers and the number of outgoing connections of the back-end. An
alternative solution with better load balancing in case of failure of one
of the back-ends, would be to have CI1-CI4 connect to both X.ci and Y.ci.
I have looked at the topologies described on the ActiveMQ site, but prefer
the one I describe above (assuming there are no problems with it) because
it is easier to set up and very flexible.
Will this topology work? Thanks in advance for your help.
--
Kind regards,
Johan Stuyts
