On Sat, Jun 4, 2011 at 2:16 AM, Thomas Hervé <thomas.he...@canonical.com> wrote: > Le 03/06/2011 04:33, Robert Collins a écrit : > > Hi Robert, thanks for sharing the ideas. I have one slightly unrelated > discussion point: > >> Rabbit has an awkward high availability story; specifically its not >> trivial to get the reliability we have out of HTTP services, This is >> partly because rabbit clusters don't distribute the queues and because >> its a more stateful and complex system than HTTP. > > I've been think about that, and I wonder how much we could cheat for > some specific workloads: if you have "short-lived" queues, I was > thinking that the application (or a smart proxy) could publish messages > to all the different RabbitMQ instances available. > > Consumers would then consume from *one* of those RabbitMQ instance, and > the others would expire after some time. You'd probably have to manage > duplicate messages on the consumer side, but that's generally the case > anyway. > > It may be a lot of work to workaround the shortcomings of RabbitMQ you > mention, though.
Indeed. Contrast this with HTTP: we configure out HTTP service point to be HA, and a node failure takes out in-progress messages only with no specific code needed in each client. With rabbit all our clients and most/all of our consumers would need to know about the HA arrangements. That seems undesirable. Now, its probably possible to write a wrapping layer that sits on amqp, takes one message, multiplies it, and sends, and conversely one that sits attached to two rabbits listening and gets advisory locks etc, or that ensures all consumers are pulling from just one rabbit. Yeouch. @Aaron burrow looks interesting. They don't /seem/ to be designing HA in (which isn't necessarily bad, but isn't necessarily good either). Some other queueing systems are around that are pretty interesting like 0mq and kestrel. I'm not sure what we should do queue wise; I'm inclined to stick with Rabbit until its either too much bother, or something massively better (e.g. massively simpler with sameish facilities, or similar complexity and more facilities (like HA)) comes along. -Rob _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
