On Thu, Aug 25, 2016 at 9:16 AM, Steven Hardy <[email protected]> wrote: > On Wed, Aug 17, 2016 at 07:20:59AM -0400, James Slagle wrote: >> On Wed, Aug 17, 2016 at 4:04 AM, Dmitry Tantsur <[email protected]> wrote: >> > However, the current gate system allows to run jobs based on files >> > affected. >> > So we can also run a scenario covering ironic on THT check/gate if >> > puppet/services/*ironic* is affected, but not in the other cases. This >> > won't >> > cover all potential failures, but it would be of great help anyway. It >> > should also run in experimental pipeline, so that it can be triggered on >> > any >> > patch. >> > >> > This is in addition to periodic jobs you're proposing, not replacing them. >> > WDYT? >> >> Using the files affected to trigger a scenario test that uses the >> affected composable service sounds like a really good idea to me. > > +1 I think this sounds like a really good idea. > > Now that we're doing almost all per-service configuration in the respective > templates and puppet profiles, this should be much easier to implement I > think so definitely +1 on giving it a go. >
So I would like to give an update about this. The work has been done to have the structure in place. I first used experimental pipeline to test the new jobs but they are now in check pipeline, as non-voting. I created a multinode jobs CI matrix here: https://github.com/openstack-infra/tripleo-ci#service-testing-matrix I encourage everyone to have a quick look at it. What is happening now? - If you submit a patch in THT (in puppet/services/ceilometer*) it will trigger scenario001 job (example with Telemetry). Same thing for Puppet profiles in puppet-tripleo. Note: with Zuul v3 we'll be able to make things more granular and specific to projects and files. We're looking for having it! - Since multinode-nonha initially created by slagle is now a bit redundant with scenarios, I'll make it lighter to test basic compute services with the less services as possible. - I'll continue to extend the scenarios complexity and start testing different backends (ie, ceph/rbd on scenario001 with Telemetry, etc), like we're doing in Puppet CI: https://github.com/openstack/puppet-openstack-integration#description - For now, we're using pingtest to test that services actually work. I guess it's good for now, but we also might want to consider Tempest sometimes. I know Tempest already runs on periodic jobs, but should we also consider it in those multinode jobs? The feedback would be valuable though but we would have to make tempest configuration composable, depending on the services we actually run (maybe using discovery, etc). Any help and feedback on this topic is highly welcome, -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
