I don't dispute that this is a useful addition to Qpid functionality, but am merely concerned that we think carefully about performance trade-offs that might arise :-)
IMHO I think the salient distinction (more to Robert's message) is that the user perceives the *messaging* component of the application architecture to have slowed. The overall end-to-end might be cheaper, but we need to mitigate the impression that Qpid performance has degraded. So, whatever the cost and the potential mitigation (recommended batch sizes etc) we need to be clear about it. So, yes - let's do it, but let's think about the various architectures we'd recommend for use with it while we do. Btw 1000/msgs sec for persistent messages of any size is not an easy win ! Regards, Marnie On 1/24/07, John O'Hara <[EMAIL PROTECTED]> wrote:
We will be supporting replayable messages. It's one of the first 'services' that we as users of other middleware have built when it doesn't exist. As Colin points out (and all the FIX users in the world) this is not a strange requirement, and our applications have had call to use the facility very routinely. A queue is a store, it is a resouce. That makes Qpid a resource manager. Obviously you shouldnt store messages in queues for ever, but it is a valid first-class persistent-store for data between applications. Performance in this type of application (to say 1000 messages per second de-queuing) is achieve by running larger transactions against the queue and committing in batches. This enables the slow part of the operation, the write to the log/journal (dept on architecture choice) to happen less frequently. John On 24/01/07, Robert Greig <[EMAIL PROTECTED]> wrote: > > On 24/01/07, Marnie McCormack <[EMAIL PROTECTED]> wrote: > > > I can see how this would help, but I do think we need to consider the > likely > > performance trade-off for persistent messaging very carefully in this > > solution. It's hard to make persistent messaging really quick anyway (or > at > > least other messaging vendors find it hard :-) so we'll need to test > > comparatively so users would understand the 'cost' of replay-ability. > > I think that users make the messages replayable in some other way if > the messaging system does not support this (e.g. slap them into a > database before putting them onto the queue is an obvious example...). > That is almost certainly more expensive in terms of performance than > the messaging system making them replayable via its store > implementation. > > RG >
