On 09/18/2014 03:45 PM, Flavio Percoco wrote:
On 09/18/2014 04:09 PM, Gordon Sim wrote:
Is the replication synchronous or asynchronous with respect to client
calls? E.g. will the response to a post of messages be returned only
once the replication of those messages is confirmed? Likewise when
deleting a message, is the response only returned when replicas of the
message are deleted?
It depends on the driver implementation and/or storage configuration.
For example, in the mongodb driver, we use the default write concern
called "acknowledged". This means that as soon as the message gets to
the master node (note it's not written on disk yet nor replicated) zaqar
will receive a confirmation and then send the response back to the
client.
So in that mode it's unreliable. If there is failure right after the
response is sent the message may be lost, but the client believes it has
been confirmed so will not resend.
This is also configurable by the deployer by changing the
default write concern in the mongodb uri using `?w=SOME_WRITE_CONCERN`[0].
[0] http://docs.mongodb.org/manual/reference/connection-string/#uri.w
So you could change that to majority to get reliable publication
(at-least-once).
[...]
From what I can see, pop provides unreliable delivery (i.e. its similar
to no-ack). If the delete call using pop fails while sending back the
response, the messages are removed but didn't get to the client.
Correct, pop works like no-ack. If you want to have pop+ack, it is
possible to claim just 1 message and then delete it.
Right, claim+delete is ack (and if the claim is replicated and
recoverable etc you can verify whether deletion occurred to ensure
message is processed only once). Using delete-with-pop is no-ak, i.e.
at-most-once.
What do you mean by 'streaming messages'?
I'm sorry, that went out wrong. I had the "browsability" term in my head
and went with something even worse. By streaming messages I meant
polling messages without claiming them. In other words, at-least-once is
guaranteed by default, whereas once-and-only-once is guaranteed just if
claims are used.
I don't see that the claim mechanism brings any stronger guarantee, it
just offers a competing consumer behaviour where browsing is
non-competing (non-destructive). In both cases you require the client to
be able to remember which messages it had processed in order to ensure
exactly once. The claim reduces the scope of any doubt, but the client
still needs to be able to determine whether it has already processed any
message in the claim already.
[...]
That marker is a sequence number of some kind that is used to provide
ordering to queries? Is it generated by the database itself?
It's a sequence number to provide ordering to queries, correct.
Depending on the driver, it may be generated by Zaqar or the database.
In mongodb's case it's generated by Zaqar[0].
Zaqar increments a counter held within the database, am I reading that
correctly? So mongodb is responsible for the ordering and atomicity of
multiple concurrent requests for a marker?
[0]
https://github.com/openstack/zaqar/blob/master/zaqar/queues/storage/mongodb/queues.py#L103-L185
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
