On 10/13/2016 11:59 PM, Steve Martinelli wrote:
On Thu, Oct 13, 2016 at 10:30 PM, Davanum Srinivas <dava...@gmail.com
<mailto:dava...@gmail.com>> wrote:


    Matt, Emilien,

    there's one more option, run jobs daily and add the output to the
    openstack health dashboard.
    example -
    
http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master
    
<http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master>


But that only runs against what is merged on master right? I think
Emilien wants to catch changes that break before they are merged.

Emilien, I think it's also worth noting which projects you intend to
make these changes to? Just "core" projects + projects that tripleO uses
(heat + ironic) + projects that have burned you in the past (OSC)? Or do
you plan on covering all projects?

Finding a balance between not enough testing, and overusing infra
resources is tricky, we should only aim for high value targets like the
ones listed above. I can't imagine this being run on all check jobs
everywhere.

It's not just overuse of infra resources. Anything that's more complicated than unit tests has > 0% chance of failure. So every new complex full stack test is going to -1 some set of good patches.

The best thing to do when there is a break is to figure out what the root cause was, and push testing back as low in the stack as possible. Was there a good unit test that could have prevented it? That's super cheap, and helps a ton going forward. Or realize that projects are using undefined behavior of other tools, and we need to define that behavior.

The deployment teams are releasing their final version of things weeks after the newton release hits. By nature there is some following time.

I kind of wonder if that hints to a better model here instead of the deployments running services from master. Instead running periodics and moving forward reference hashes every day if tests pass, and not if they fail. That would let deployment tools automatically move forward, quite close to master, but not get tightly coupled into every change.

Because I do get concerned for deciding that every developer needs to understand tripleo, and ansible, and fuel and the configuration decisions they made, the bugs they have, the way their services are deployed, to be able to land any patches. Because I think that will further reduce the pool of developers that can contribute.

        -Sean

--
Sean Dague
http://dague.net

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to