I don't think the radosgw supports bulk yet. They haven't for a very long time. :/
Thanks, Kevin ________________________________________ From: Dan Prince [dpri...@redhat.com] Sent: Thursday, January 07, 2016 9:03 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [TripleO] Is Swift a good choice of database for the TripleO API? On Tue, 2015-12-22 at 15:36 +0000, Dougal Matthews wrote: > Hi all, > > This topic came up in the 2015-12-15 meeting[1], and again briefly > today. > After working with the code that came out of the deployment library > spec[2] I > had some concerns with how we are storing the templates. > > Simply put, when we are dealing with 100+ files from tripleo-heat- > templates > how can we ensure consistency in Swift without any atomicity or > transactions. > I think this is best explained with a couple of examples. > > - When we create a new deployment plan (upload all the templates to > swift) > how do we handle the case where there is an error? For example, if > we are > uploading 10 files - what do we do if the 5th one fails for some > reason? > There is a patch to do a manual rollback[3], but I have concerns > about > doing this in Python. If Swift is completely inaccessible for a > short > period the rollback wont work either. Would Swift's Bulk middleware help us here: https://github.com/openstack/swift/blob/master/swift/common/middleware/ bulk.py We don't currently have that middleware enabled but we certainly could try it out. NOTE the "Expand tar files into a swift account" feature... Dan > > - When deploying to Heat, we need to download all the YAML files > from Swift. > This can take a couple of seconds. What happens if somebody starts > to > upload a new version of the plan in the middle? We could end up > trying to > deploy half old and half new files. We wouldn't have a consistent > view of > the database. > > We had a few suggestions in the meeting: > > - Add a locking mechanism. I would be concerned about deadlocks or > having to > lock for the full duration of a deploy. > - Store the files in a tarball (so we only deal with one file). I > think we > could still hit issues with multiple operations unless we > guarantee that > only one instance of the API is ever running. > > I think these could potentially be made to work, but at this point > are we > using the best tool for the job? > > For alternatives, I see a can think of a couple of options: > > - Use a database, like we did for Tuskar and most OpenStack API's do. > - Invest time in building something on Swift. > - Glance was transitioning to be a Artifact store. I don't know the > status of > this or if it would meet out needs. > > Any input, ideas or suggestions would be great! > > Thanks, > Dougal > > > [1]: http://eavesdrop.openstack.org/meetings/tripleo/2015/tripleo.201 > 5-12-15-14.03.log.html#l-89 > [2]: https://specs.openstack.org/openstack/tripleo-specs/specs/mitaka > /tripleo-overcloud-deployment-library.html > [3]: https://review.openstack.org/#/c/257481/ > _____________________________________________________________________ > _____ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > cribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev