On 08/05/2015 02:33 PM, Ryan Hallisey wrote: > Tagging kolla so the kolla community also sees it. > Pardon the top posting. > > -Ryan > > ----- Original Message ----- > From: "Dan Prince" <[email protected]> > To: "openstack-dev" <[email protected]> > Sent: Wednesday, August 5, 2015 2:29:13 PM > Subject: [openstack-dev] [TripleO] [Puppet] Deploying OpenStack with Puppet > modules on Docker with Heat > > Hi, > > There is a lot of interest in getting support for container based > deployment within TripleO and many different ideas and opinions on how > to go about doing that. > > One idea on the table is to use Heat to help orchestrate the deployment > of docker containers. This would work similar to our tripleo-heat > -templates implementation except that when using docker you would swap > in a nested stack template that would configure containers on > baremetal. We've even got a nice example that shows what a > containerized TripleO overcloud might look like here [1]. The approach > outlines how you might use kolla docker containers alongside of the > tripleo-heat-templates to do this sort of deployment. > > This is all cool stuff but one area of concern is how we do the actual > configuration of the containers. The above implementation relies on > passing environment variables into kolla built docker containers which > then self configure all the required config files and start the > service. This sounds like a start... but creating (and maintaining) > another from scratch OpenStack configuration tool isn't high on my list > of things to spend time on. Sure there is already a kolla community > helping to build and maintain this configuration tooling (mostly > thinking config files here) but this sounds a bit like what tripleo > -image-elements initially tried to do and it turns out there are much > more capable configuration tools out there. > > Since we are already using a good bit of Puppet in tripleo-heat > -templates the idea came up that we would try to configure Docker > containers using Puppet. Again, here there are several ideas in the > Puppet community with regards to how docker might best be configured > with Puppet. Keeping those in mind we've been throwing some ideas out > on an etherpad here [2] that describes using Heat for orchestration, > Puppet for configuration, and Kolla docker images for containers. > > A quick outline of the approach is: > > -Extend the heat-container-agent [3] that runs os-collect-config and > all the required hooks we require for deployment. This includes docker > -compute, bash scripts, and Puppet. NOTE: As described in the etherpad > I've taken to using DIB to build this container. I found this to be > faster from a TripleO development baseline. > > -To create config files the heat-container-agent would run a puppet > manifest for a given role and generate a directory tree of config files > (/var/lib/etc-data for example).
I have a few questions: * when do you run puppet? before starting the container so we can generate a configuration file? * so iiuc, Puppet is only here to generate OpenStack configuration files and we noop all other operations. Right? * from a Puppet perspective, I really prefer this approach: https://review.openstack.org/#/c/197172/ where we assign tags to resources so we can easily modify/drop Puppet resources using our modules. What do you think (for long term)? * how do you manage multiple configuration files? (if a controller is running multiple nova-api containers with different configuration files? Once I understand a bit more where we go, I'll be happy to help to make it happen in our modules, we already have folks deploying our modules with containers, I guess we can just talk and collaborate here. Also, I'll be interested to bringing containers support in our CI, but that's a next step :-) Thanks Dan for this work, > > -We then run a docker-compose software deployment that mounts those > configuration file(s) into a read only volume and uses them to start > the containerized service. > > The approach could look something like this [4]. This nice thing about > this is that it requires no modification to OpenStack Puppet modules. > We can use those today, as-is. Additionally, although Puppet runs in > the agent container we've created a mechanism to set all the resources > to noop mode except for those that generate config files. And lastly, > we can use exactly the same role manifest for docker that we do for > baremetal. Lots of re-use here... and although we are disabling a lot > of Puppet functionality in setting all the non-config resources to noop > the Kolla containers already do some of that stuff for us (starting > services, etc.). > > ---- > > All that said (and trying to keep this short) we've still got a bit of > work to do around wiring up externally created config files to kolla > build docker containers. A couple of issues are: > > -The external config file mechanism for Kolla containers only seems to > support a single config file. Some services (Neutron) can have multiple > files. Could we extend the external config support to use multiple > files? > > -If a service has multiple files kolla may need to adjust its service > startup script to use multiple files. Perhaps a conf.d approach would > work here? > > -We are missing published version of some key kolla containers. Namely > openvswitch and the neutron-openvswitch-agent for starters but I'd also > like to have a Ceilometer agent and SNMP agent container as well so we > have feature parity with the non-docker compute role. > > Once we have solutions for the above I think we'll be very close to a > fully dockerized compute role with TripleO heat templates. From there > we can expand the idea to cover other roles within the tripleo-heat > -templates too. > > I'll stop there for now. Any ideas and thoughts appreciated. > > Dan > > ----- > > [1] https://review.openstack.org/#/c/178840/ (Containerized TripleO > Overcloud.) > [2] https://etherpad.openstack.org/p/tripleo-docker-puppet > [3] http://git.openstack.org/cgit/openstack/heat > -templates/log/hot/software-config/heat-container-agent > [4] https://review.openstack.org/#/c/209505/ (Docker compute role > configured via Puppet) > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Emilien Macchi
signature.asc
Description: OpenPGP digital signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
