Greetings, The Zaqar team has been quiet in the last cycle but that doesn't mean it's gone. This email is to share with you all what we've been focused on during Kilo.
The team has been working on 3 main areas (order does not reflect priority/importance): * Relaxing some of the storage constraints by relying more on storage capabilities and less on the guarantees. [0] * Implement notifications to fulfill that missing part of the service's goals. [1] * Work on a non-wsgi transport[2] Each of the above areas required important changes in the service that I'll try to sumarize now: Relaxing Storage Constraints ============================ As some of you know, the service has support for something we call "flavors". These flavors difer from what we know as flavors in services like Nova. The flavors in Zaqar, represent the type of storage a set of messages will be stored into. Every flavor in Zaqar has a set of capabilities. These capabilities represent what the storage can/cannot do. For example, durability is considered a capability and so are high-throughput and claims. As of Juno, these capabilities were manually created but this has changed. The capabilities are now introspected from the pools asigned to the flavors and they are now a read-only field. In addition, as part of this work, FIFO has been made optional and it can now be enabled through the pool. [3] To know a bit more about flavors and pools, please read this post. [4] Implementing notifications ========================== Fei Long Wang (flwang) has done an amazing job on getting this done. The feature itself was quite big and it required patches in several places. If you're interested in the details of the feature, please, read the spec[1]. If you're going to use notifications, please, bare in mind that the feature itself is quite new and it requires some extra testing (especiall scale tests ;). Non persistent transport ======================== Victoria Martinez de la Cruz (vkmc) has dedicated efforts to this feature. She's done an amazing research on this field to find the best way to make this work with Zaqar. During the cycle several websocket libraries were examined and we ended up choosing atobhan. Likewise, the protocol went through some iterations until we finally reached an agreement. This feature, however, didn't make it entirely in Kilo. Part of it will be completed during Liberty. To be precise, it's possible to manage queue's and connect to Zaqar using websockets but it's still not possible to fully operate the API. We felt that it was not right to rush this in at its current state and that it'd be a good thing to have part of the feature as a preview for some folks to play with it. Move QueueController to the control plane ========================================= This might be confusing for some folks that are not familiar with Zaqar's architecture but I think it's also worth mentioning. TL;DR: Zaqar has 2 data planes. 1 used for the actual data (messages) and one used for the "control" - metadata- data (subscriptions information, flavors, etc). These 2 planes can either use the same DB or a different one. Queue's used to live in the data plane but we've moved them to the control plane. The main reason is that queue's are a lazy resource in Zaqar and you don't really need to create them manually. The only reason you'd create a queue manually is to set some metadata for it (assign a flavor to a queue, for example). Since this is considered to be metadata, we got to the conclusion there was no value on keeping queue's in the data plane. This move brings other benefits like saving space on the data store, avoid hitting the data store for queue's lookups and management, pick better stores for each plane, etc. I'd like to thank Shaifali Agrawal for her persistence and hard work on this task. It was not an easy task, it required internal refactoring, configuration changes and lots of fights with gerrit. Thanks again! The above changes, since they required breaking backwards compatibily, have been implemented in a new version of the API, v2. The v2 follows closely what existed already in v1.1 but it also has support for these new features and the breaking changes I've mentioned already. Unfortunately, as many of you know, our team has shrunk and each of us work on other projects too. This means that there's little to none client support for the above mentioned features. However, we'll be working on bringing the client up-to-speed. The team spent the entire cycle heads-down silently working on these tasks in the best way possible. For the next cycles, I believe the main focus will be in improving the adoption of the service and improving UX besides fixing bugs, obviously. Hope you find this summary useful and please, don't hesitate to shoot questiongs if you've any. Flavio [0] https://github.com/openstack/zaqar-specs/blob/master/specs/kilo/approved/storage-capabilities.rst [1] https://github.com/openstack/zaqar-specs/blob/master/specs/kilo/approved/notification-api.rst [2] https://github.com/openstack/zaqar-specs/blob/master/specs/kilo/approved/persistent-transport.rst [3] https://github.com/openstack/zaqar-specs/blob/master/specs/kilo/approved/fifo-optional.rst [4] http://blog.flaper87.com/post/zaqar-pools-explained/ [5] https://blueprints.launchpad.net/zaqar/+spec/split-data-and-control-plane -- @flaper87 Flavio Percoco
pgpJHud72T2XN.pgp
Description: PGP signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev