Hi all, This topic came up in the 2015-12-15 meeting[1], and again briefly today. After working with the code that came out of the deployment library spec[2] I had some concerns with how we are storing the templates.
Simply put, when we are dealing with 100+ files from tripleo-heat-templates how can we ensure consistency in Swift without any atomicity or transactions. I think this is best explained with a couple of examples. - When we create a new deployment plan (upload all the templates to swift) how do we handle the case where there is an error? For example, if we are uploading 10 files - what do we do if the 5th one fails for some reason? There is a patch to do a manual rollback[3], but I have concerns about doing this in Python. If Swift is completely inaccessible for a short period the rollback wont work either. - When deploying to Heat, we need to download all the YAML files from Swift. This can take a couple of seconds. What happens if somebody starts to upload a new version of the plan in the middle? We could end up trying to deploy half old and half new files. We wouldn't have a consistent view of the database. We had a few suggestions in the meeting: - Add a locking mechanism. I would be concerned about deadlocks or having to lock for the full duration of a deploy. - Store the files in a tarball (so we only deal with one file). I think we could still hit issues with multiple operations unless we guarantee that only one instance of the API is ever running. I think these could potentially be made to work, but at this point are we using the best tool for the job? For alternatives, I see a can think of a couple of options: - Use a database, like we did for Tuskar and most OpenStack API's do. - Invest time in building something on Swift. - Glance was transitioning to be a Artifact store. I don't know the status of this or if it would meet out needs. Any input, ideas or suggestions would be great! Thanks, Dougal [1]: http://eavesdrop.openstack.org/meetings/tripleo/2015/tripleo.2015-12-15-14.03.log.html#l-89 [2]: https://specs.openstack.org/openstack/tripleo-specs/specs/mitaka/tripleo-overcloud-deployment-library.html [3]: https://review.openstack.org/#/c/257481/
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
