Hi Pamod. Thanks for bringing it up and I agree. To add to that, as suggested by Srinath the ideal solution here would be to map MQTT QOS levels directly to AMQP exchange types.
Given that, we could derive that QOS 0 is the exact non-durable topic scenario running with consumers on auto_ack mode. Thanks On Tue, Nov 18, 2014 at 3:22 PM, Pamod Sylvester <[email protected]> wrote: > Hi Hasitha, > > In the Andes Kernal level we need to have a mechanism to consider acks for > MQTT messages, for QOS 1 and 2 the acks will be considered and will be > retried if unacked. Specially in the case of QOS 1 type of messages. > > Thanks, > Pamod > > On Tue, Nov 18, 2014 at 3:16 PM, Hasitha Amal De Silva <[email protected]> > wrote: > >> Hi All, >> >> Given our current messaging model, all messages go through a durable >> store, and during non-durable topics, we use only one storage queue for >> multiple consumers pointed at a single node for the same topic. >> >> As discussed with HasithaH, This brings up the following concerns when >> working with non-durable topics. >> >> *1. Consumer can receive messages queued before the consumer was started* >> >> Given that there are 2 consumers C1,C2 on MB node N1 for non-durable >> topic T (Lets call the storage queue T_N1) >> >> Scenario : Publishers will keep publishing to C2 and C1 while C1 closes. >> Now, since C2 is active, it will still receive messages. Assume that the >> publish rate is high and that theres a lot of messages still in the storage >> queue. At this moment, another consumer C3 joins N1. As per current >> implementation, C3 will receive the previously queued messages. >> >> Is this wrong ? Should a non-durable topic consumer only receive messages >> that have been added after the consumer was started ? >> >> The "right" i see here is that both C2 and C3 are expecting messages from >> the same topic, and thus they deserve to receive past messages of that >> topic, if not yet delivered, while the server is active. This is also what >> i understood from the spec [1][2]. >> >> *2. Can we fire-and-forget non-durable topics without waiting for client >> acks ? * >> >> As per 2.2.0, our model on non-durable topics was to simply clear >> messages once they are handed over to the delivery thread. But since we >> have mandated storage queues for non-durable topics in MB 3.0.0, we now >> handle these messages based on acks, rejects from the client. (e.g. >> rejected messages can be sent to a second consumer) >> >> As I understood from the amqp spec 0_9_1, a message can only be >> fired-and-forgotten if the consumer's acknowledgement mode is set to >> "automatic" [3]. This is strictly under control of the consumer. This could >> allow a non-durable topic consumer to set "explicit" acknowledgement mode, >> which would require proper ack/reject handling. >> >> How should we handle this ? >> >> - Can we instruct clients to always set the ack mode to "auto" on >> non-durable consumers -> and then fire-and forget messages ? >> >> or >> >> - Should we have two logical flows to handle both scenarios on >> non-durable topics ? >> >> >> Appreciate your thoughts on this from use-case perspective too, since >> this a slightly gray area in the spec :) >> >> -------------------------------------------------------------------------------------------------------------------- >> >> Following excerpts were taken from amqp spec 0_9_1 : >> https://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf >> >> >> [1] : *3.1.3 Exchanges *: "Exchanges may be durable, temporary, or >> auto-deleted. Durable exchanges last until they are deleted. Temporary >> exchanges last until the server shuts-down. Auto-deleted exchanges last >> until they are no longer used." >> >> [2] : *3.1.4 Message Queues* : "Message queues may be durable, >> temporary, or auto-deleted. Durable message queues last until they are >> deleted. Temporary message queues last until the server shuts-down. >> Auto-deleted message queues last until they are no longer used."..."A >> message routed to a message queue is never sent to more than one client >> unless it is being resent after a failure or rejection." >> >> [3] *3.1.8 Acknowledgements* : "An acknowledgement is a formal signal >> from the client application to a message queue that it has successfully >> processed a message. There are two possible acknowledgement models: >> >> 1. Automatic, in which the server removes a content from a message >> queue as soon as it delivers it to an application (via the Deliver or >> Get-Ok methods). >> 2. Explicit, in which the client application must send an Ack method >> for each message, or batch of messages, that it has processed. >> >> The client layers can themselves implement explicit acknowledgements in >> different ways, e.g. as soon as a message is received, or when the >> application indicates that it has processed it. These differences do not >> affect AMQP or interoperability." >> >> >> -- >> Cheers, >> >> Hasitha Amal De Silva >> Software Engineer >> Mobile : 0772037426 >> Blog : http://devnutshell.tumblr.com/ >> WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. ) >> > > > > -- > *Pamod Sylvester * > *Senior Software Engineer * > Integration Technologies Team, WSO2 Inc.; http://wso2.com > email: [email protected] cell: +94 77 7779495 > -- Cheers, Hasitha Amal De Silva Software Engineer Mobile : 0772037426 Blog : http://devnutshell.tumblr.com/ WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. )
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
