It looks like dox(1) is really cool, since it will allow us to start using Docker.
I started a very experimenting patch in dox to support Puppet unit tests: https://review.openstack.org/#/c/119943 (very early stage) The idea would be: * patch dox to support Puppet unit testing (as we did in virtual env with bundle) within containers * change Puppet jobs (for one module to try) to use dox as non-voting (to ensure we don't have regression somewhere) * When it's alright, patch puppet jobs to use dox with voting for all modules. * integrate new scenarios for functional testing by using beaker & dox. Any thoughts? (1) https://github.com/stackforge/dox + http://article.gmane.org/gmane.comp.cloud.openstack.devel/34675 Emilien Macchi On 09/08/2014 01:21 PM, Emilien Macchi wrote: > Hello Openstack Infra Team, > > > Background > ======= > > It has been 3 years that we have been developing Puppet modules to > install OpenStack, and we know these modules are largely adopted by > many users. > All modules are located on StackForge (1) and some of them are > distributed on Puppet Forge. > Some statistics (2) show us the important adoption, which represent ~ > 42% of current deployments (Source: OpenStack Foundation). > We've reached a point where we can't continue without real integration > testing if we want to stabilize & bring more dev/users to our modules. > > > Current situation > ========== > > OpenStack Puppet modules support rspec & unit tests which is not > enough since there is no functional testing. > Currently, only puppet-ceph (3) is doing integration testing (4) by > using Vagrant, puppet-rspec-system and gerritexec (5). > A discussion has been started on the Puppet OpenStack mailing list (6). > As you can see, there has already been some initial proof of concept > work done by Spencer to make beaker-rspec work with the > puppet-openstack modules: > https://review.openstack.org/#/c/102020 > <https://review.openstack.org/#/c/102020> # puppet-nova > https://review.openstack.org/#/c/102071 > <https://review.openstack.org/#/c/102071> # openstack-infra/config > (patch already in discussion with -infra team) > > > Design proposal > ========== > > Since puppet-rspec-system is going to be retired (8), it's suggested > to use beaker-rspec. > A new bot would be created "Puppet OpenStack CI" which would put +1 or > -1 depending if the integration pass or not. For starting, the bot > won't be part of voting process. > The goal would be of course to bootstrap basic tests for all modules > and then call to contributors wanting to support most of deployments > use-cases by using puppet classes. > > > Blockers > ===== > > We need resources to handle the CI bot system. puppet-ceph only uses a > single server + vagrant, but in the case of OpenStack modules, I'm not > sure that would be sufficient regarding to the number of patches per day. > I would like to ensure that OpenStack Infra could handle vagrant on CI > nodes. > > > Steps & goals > ========= > > * First, we need to agree with -infra team at how we handle CI > resources (vagrant ?). > * Patch Jenkins jobs (already WIP) for non-voting jobs in some Puppet > modules (core projects to start: nova, neutron, ...). > * Push some patches in puppet-* modules with basic tests. The goal > here is to get feedback from this design. > * Continue with more modules (non-core) and more testing scenarios. > * After stabilization, make the CI bot voting. > > > We really need to make progress on this topic if we want more adoption > of Puppet modules in OpenStack community. I'm thinking about Puppet > elements in TripleO (some work has already been done, but not yet in > TripleO though) (8). > Having this CI would help people to stop forking the modules and > improve the trust at the work done. > By the way, this topic will be one of main topics at the next > OpenStack summit (9). > > > On behalf of Puppet OpenStack team, we are waiting for your insights, > Thanks. > > (1) https://github.com/stackforge?page=1&query=puppet > (2) http://getcloudify.org/2014/05/19/openstack-statistics.html > (3) https://github.com/stackforge/puppet-ceph > (4) > https://github.com/stackforge/puppet-ceph/blob/master/README.md#integration-tests > (5) https://pypi.python.org/pypi/gerritexec > (6) > htt6s://groups.google.com/a/puppetlabs.com/forum/#!topic/puppet-openstack/Mr2wNC4HuZU > (7) https://github.com/puppetlabs/rspec-system-puppet#retired > (8) https://github.com/agroup/tripleo-puppet-elements > (9) https://etherpad.openstack.org/p/puppet-paris-summit-proposal > > -- > Emilien Macchi
signature.asc
Description: OpenPGP digital signature
_______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
