Hi All, Given our current messaging model, all messages go through a durable store, and during non-durable topics, we use only one storage queue for multiple consumers pointed at a single node for the same topic.
As discussed with HasithaH, This brings up the following concerns when working with non-durable topics. *1. Consumer can receive messages queued before the consumer was started* Given that there are 2 consumers C1,C2 on MB node N1 for non-durable topic T (Lets call the storage queue T_N1) Scenario : Publishers will keep publishing to C2 and C1 while C1 closes. Now, since C2 is active, it will still receive messages. Assume that the publish rate is high and that theres a lot of messages still in the storage queue. At this moment, another consumer C3 joins N1. As per current implementation, C3 will receive the previously queued messages. Is this wrong ? Should a non-durable topic consumer only receive messages that have been added after the consumer was started ? The "right" i see here is that both C2 and C3 are expecting messages from the same topic, and thus they deserve to receive past messages of that topic, if not yet delivered, while the server is active. This is also what i understood from the spec [1][2]. *2. Can we fire-and-forget non-durable topics without waiting for client acks ? * As per 2.2.0, our model on non-durable topics was to simply clear messages once they are handed over to the delivery thread. But since we have mandated storage queues for non-durable topics in MB 3.0.0, we now handle these messages based on acks, rejects from the client. (e.g. rejected messages can be sent to a second consumer) As I understood from the amqp spec 0_9_1, a message can only be fired-and-forgotten if the consumer's acknowledgement mode is set to "automatic" [3]. This is strictly under control of the consumer. This could allow a non-durable topic consumer to set "explicit" acknowledgement mode, which would require proper ack/reject handling. How should we handle this ? - Can we instruct clients to always set the ack mode to "auto" on non-durable consumers -> and then fire-and forget messages ? or - Should we have two logical flows to handle both scenarios on non-durable topics ? Appreciate your thoughts on this from use-case perspective too, since this a slightly gray area in the spec :) -------------------------------------------------------------------------------------------------------------------- Following excerpts were taken from amqp spec 0_9_1 : https://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf [1] : *3.1.3 Exchanges *: "Exchanges may be durable, temporary, or auto-deleted. Durable exchanges last until they are deleted. Temporary exchanges last until the server shuts-down. Auto-deleted exchanges last until they are no longer used." [2] : *3.1.4 Message Queues* : "Message queues may be durable, temporary, or auto-deleted. Durable message queues last until they are deleted. Temporary message queues last until the server shuts-down. Auto-deleted message queues last until they are no longer used."..."A message routed to a message queue is never sent to more than one client unless it is being resent after a failure or rejection." [3] *3.1.8 Acknowledgements* : "An acknowledgement is a formal signal from the client application to a message queue that it has successfully processed a message. There are two possible acknowledgement models: 1. Automatic, in which the server removes a content from a message queue as soon as it delivers it to an application (via the Deliver or Get-Ok methods). 2. Explicit, in which the client application must send an Ack method for each message, or batch of messages, that it has processed. The client layers can themselves implement explicit acknowledgements in different ways, e.g. as soon as a message is received, or when the application indicates that it has processed it. These differences do not affect AMQP or interoperability." -- Cheers, Hasitha Amal De Silva Software Engineer Mobile : 0772037426 Blog : http://devnutshell.tumblr.com/ WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. )
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
