Jonathan What netsplit scenarios do you anticipate?
alexis On Tue, Nov 8, 2011 at 4:26 PM, Jonathan Oliver <[email protected]> wrote: > Since the introduction of better clustering support and message queue > failover in v2.6, my company is starting to consider moving to RabbitMQ. > But we had a few questions about the roadmap. > We are looking to ensure that a message is only considered received by an > exchange from a producing app when at least two physical instances of Rabbit > acknowledge the receipt of the message--similar to Riak's method. > Furthermore, we are looking to ensure that a message is considered consumed > (ack'd) when when at least one physical instance receives the ack, which may > result in a message being delivered more than once. Essentially, we'd like > to propose a configuration where Rabbit can fall on the AP side of CAP > instead of the CA side. > I should mention that we're fully aware that the above scenario may cause a > message to be pushed to consumers more than once and how messages may get > out of order. In general, however, these concepts are very much a part of > message queuing system guarantees--that of at-least-once delivery where > order is not strictly guaranteed. If we needed at-most-once delivery and > strict ordering, we'd use RPC. In many regards, this is similar to the > Amazon SQS architecture with a few key differences--one of the most > important being stateful connections and "push" capabilities between the > server and connected consumers which is where Amazon's SQS offering falls > apart--that and message size restrictions. > I can see that many systems really want to ensure at-most-once delivery (or > close to it) and how the requirement to re-order messages makes things > difficult, but by relaxing the constraints (at least in some > configurations), it enables so much more. For example, if an instance of > Rabbit goes offline under the current HA configuration, when it rejoins the > cluster the queues are wiped clean. Yuck. So much for guaranteed delivery. > While under some kind of partition-tolerant configuration, the queues > wouldn't need to be cleaned. Instead, the consuming application would be > responsible for message idempotency. > At the same time, I realize that you guys might be bumping against > restrictions in the AMQP standard, for example multiple consumers receiving > copies of the message where one consumer ack's the message, while another > nack's the message. But that's something where the messaging system can't > determine the proper course of action without knowing the business context. > In that case, I would imaging it should present that nack back to the > producer as it currently does. > The bottom line in my question and proposal really has to do with message > loss. If the idea is to not lose messages, I can see a partition-tolerant > configuration bringing a lot to the table and where the consuming > application must be resilient to duplicate messages and ordering, thus > freeing up RabbitMQ from having to worry so much about that. > _______________________________________________ > rabbitmq-discuss mailing list > [email protected] > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss > > -- You received this message because you are subscribed to the Google Groups "rabbitmq-discuss" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rabbitmq-discuss?hl=en.
