On Wed, 2014-09-10 at 14:51 +0200, Thierry Carrez wrote: > Flavio Percoco wrote: > > [...] > > Based on the feedback from the meeting, the current main concern is: > > > > - Do we need a messaging service with a feature-set akin to SQS+SNS? > > [...] > > I think we do need, as Samuel puts it, "some sort of durable > message-broker/queue-server thing". It's a basic application building > block. Some claim it's THE basic application building block, more useful > than database provisioning. It's definitely a layer above pure IaaS, so > if we end up splitting OpenStack into layers this clearly won't be in > the inner one. But I think "IaaS+" basic application building blocks > belong in OpenStack one way or another. That's the reason I supported > Designate ("everyone needs DNS") and Trove ("everyone needs DBs"). > > With that said, I think yesterday there was a concern that Zaqar might > not fill the "some sort of durable message-broker/queue-server thing" > role well. The argument goes something like: if it was a queue-server > then it should actually be built on top of Rabbit; if it was a > message-broker it should be built on top of postfix/dovecot; the current > architecture is only justified because it's something in between, so > it's broken. > > I guess I don't mind that much zaqar being "something in between": > unless I misunderstood, exposing extra primitives doesn't prevent the > "queue-server" use case from being filled. Even considering the > message-broker case, I'm also not convinced building it on top of > postfix/dovecot would be a net win compared to building it on top of > Redis, to be honest.
AFAICT, this part of the debate boils down to the following argument: If Zaqar implemented messaging-as-a-service with only queuing semantics (and no random access semantics), it's design would naturally be dramatically different and "simply" implement a multi-tenant REST API in front of AMQP queues like this: https://www.dropbox.com/s/yonloa9ytlf8fdh/ZaqarQueueOnly.png?dl=0 and that this architecture would allow for dramatically improved throughput for end-users while not making the cost of providing the service prohibitive to operators. You can't dismiss that argument out-of-hand, but I wonder (a) whether the claimed performance improvement is going to make a dramatic difference to the SQS-like use case and (b) whether backing this thing with an RDBMS and multiple highly available, durable AMQP broker clusters is going to be too much of a burden on operators for whatever performance improvements it does gain. But the troubling part of this debate is where we repeatedly batter the Zaqar team with hypotheses like these and appear to only barely entertain their carefully considered justification for their design decisions like: https://wiki.openstack.org/wiki/Frequently_asked_questions_%28Zaqar%29#Is_Zaqar_a_provisioning_service_or_a_data_API.3F https://wiki.openstack.org/wiki/Frequently_asked_questions_%28Zaqar%29#What_messaging_patterns_does_Zaqar_support.3F I would like to see an SQS-like API provided by OpenStack, I accept the reasons for Zaqar's design decisions to date, I respect that those decisions were made carefully by highly competent members of our community and I expect Zaqar to evolve (like all projects) in the years ahead based on more real-world feedback, new hypotheses or ideas, and lessons learned from trying things out. Mark. _______________________________________________ OpenStack-dev mailing list OpenStackemail@example.com http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev