On 4 Feb 2009, at 22:50, Robert Greig wrote:
The big difficulty we have is a hard requirement for in-order message
processing. This unfortunately rules out load balancing from a
queue across
active-active client pairs. We'd rather just route incoming msgs to a
store-and-forward topic, and accept a small processing pause during
app node
failover.
Do you have the requirement for total ordering or is it just a partial
ordering, i.e. you need to process requests for a given order id in
the right order (i.e. new/amend/cancel for a given order id)?
It's a partial ordering, per instrument. Ie, orders on the same
instrument need to be processed sequentially.
Once a message has been read from the topic, it is written into the
TC
cache. At that point we can failover processing to a different node
without
any message or data loss.
Does TC now support memory replication between the server nodes?
Previously it required shared storage which meant the performance
sucked.
I can imagine. It now does in-memory replication between a master &
slave. They still have the shared storage mode but it's basically
deprecated these days I believe.
It also does notification of client leave/join (to remaining clients).
The general idea is we use qpid to split messages between TC client
nodes (for data locality), and TC notifications for HA (another node
can take over processing of a given stream of orders).
Absolutely - and we haven't ruled anything out yet. We're going to
start
with the lowest cost option, and if it doesn't work out, then have a
rethink.
What are your throughput and latency requirements?
This is an interesting domain since it blurs the distinction between
messaging and in-memory data grids.
Most definitely. Like most things, 'as much throughput as fast as
possible' - according to the business at least! qpid seems more than
capable of handling our throughput requirements. We're currently
investigating a few application structure scenarios to see which one
meets our latency/load balancing needs the best.
Cheers,
Andrew
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
