On Fri, Oct 14, 2016 at 7:33 AM, Sean Dague <s...@dague.net> wrote: > On 10/13/2016 11:59 PM, Steve Martinelli wrote: >> >> On Thu, Oct 13, 2016 at 10:30 PM, Davanum Srinivas <dava...@gmail.com >> <mailto:dava...@gmail.com>> wrote: >> >> >> Matt, Emilien, >> >> there's one more option, run jobs daily and add the output to the >> openstack health dashboard. >> example - >> >> http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master >> >> <http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master> >> >> >> But that only runs against what is merged on master right? I think >> Emilien wants to catch changes that break before they are merged. >> >> Emilien, I think it's also worth noting which projects you intend to >> make these changes to? Just "core" projects + projects that tripleO uses >> (heat + ironic) + projects that have burned you in the past (OSC)? Or do >> you plan on covering all projects? >> >> Finding a balance between not enough testing, and overusing infra >> resources is tricky, we should only aim for high value targets like the >> ones listed above. I can't imagine this being run on all check jobs >> everywhere. > > > It's not just overuse of infra resources. Anything that's more complicated > than unit tests has > 0% chance of failure. So every new complex full stack > test is going to -1 some set of good patches. > > The best thing to do when there is a break is to figure out what the root > cause was, and push testing back as low in the stack as possible. Was there > a good unit test that could have prevented it? That's super cheap, and helps > a ton going forward. Or realize that projects are using undefined behavior > of other tools, and we need to define that behavior. > > The deployment teams are releasing their final version of things weeks after > the newton release hits. By nature there is some following time. > > I kind of wonder if that hints to a better model here instead of the > deployments running services from master. Instead running periodics and > moving forward reference hashes every day if tests pass, and not if they > fail. That would let deployment tools automatically move forward, quite > close to master, but not get tightly coupled into every change. > > Because I do get concerned for deciding that every developer needs to > understand tripleo, and ansible, and fuel and the configuration decisions > they made, the bugs they have, the way their services are deployed, to be > able to land any patches. Because I think that will further reduce the pool > of developers that can contribute.
I explicitly mentioned the opposite in the proposal, let me copy/paste it again: "We don't expect developers to investigate why new CI jobs would fail (ex: a Nova developer to look at TripleO CI job), it would be unfair. Just some horizontal communication would be enough, IRC or email, to inform that a patch might break a CI job outside devstack." If a CI job is breaking because of a patch, deployment tools folks would look at the CI job and tell to the project why it fails, who would decide if whether or not it's a blocker (ie: indeed, the patch is breaking backward compatibility, or no, the patch is good, installer has to be updated, etc). I hope it's clear that we don't expect project to spend time on this thing and of course we don't want to reduce the pool of devs here. -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev