On Sep 26, 2014, at 2:27 PM, Sean Dague <s...@dague.net> wrote:

> As we've been talking about the test disaggregation the hamster wheels
> in the back of my brain have been churning on the library testing
> problem I think we currently have. Namely, openstack components mostly
> don't rely on released library versions, they rely on git master of
> them. Right now olso and many clients are open masters (their
> stable/juno versions are out), but we've still not cut RCs on servers.
> So we're actually going to have a time rewind event as we start cutting
> stables.
> We did this as a reaction to the fact that library releases were often
> cratering the world. However, I think the current pattern leads us into
> a much more dangerous world where basically the requirements.txt is invalid.
> So here is the particular unwind that I think would be useful here:
> 1) Change setup_library in devstack to be able to either setup the
> library from git or install via pip. This would apply to all libraries
> we are installing from oslo, the python clients, stackforge, etc.
> Provide a mechanism to specify LIBRARIES_FROM_GIT (or something) so that
> you can selectively decide to use libraries from git for development
> purposes.
> 2) Default devstack to use pip released versions.

These first 2 suggestions seem good. It makes our test configuration more 
accurate, and frees the Oslo team to have our dev cycle a little out of phase 
without worrying about breaking things. It means app developers will have to 
wait for fixes and features longer than they would otherwise, but Thierry has 
improved the automation for release tracking so we could release more often.

> 3) Change the job definition on the libraries to test against devstack
> in check, not in gate. The library teams can decide if they want their
> forward testing to be voting or not, but this is basically sniff testing
> that when they release a new library they won't ruin the world for
> everyone else.

The other suggestions only work if we also have some job that does still use 
the source version of the libraries. A voting check job is good. Why would we 
not also do the test in the gate?

> 4) If a ruin the world event happens, figure out how to prevent that
> kind of event in local project testing, unit or functional. Basically an
> unknown contract was broken. We should bring that contract back into the
> project itself, or yell at the consuming project about why they were
> using code in a crazy pants way.
> Additionally, I'd like us to consider: No more alpha libraries. The
> moment we've bumped global requirements in projects we've actually
> released these libraries to production, as people are CDing the servers.
> We should just be honest about that and just give things a real version.
> Version numbers are cheap.

Anyone following trunk is already expecting some pain, but the folks that 
follow the stable branches do it because we call them stable. We have been 
using alpha versions as a way to avoid throwing development versions of 
libraries into the stable branch test and production environments before they 
have had some runtime against master. It’s not so much an indication of the 
quality of the content of the package as a hack taking advantage of pip 
behaviors that don’t install pre-release packages unless specifically told do 
so, and a signal to distros that they don’t necessarily have to package those 
releases (I don’t know if they’re receiving that signal or not). If we stop 
using alpha version numbers, we need some other way to avoid having development 
releases end up in the stable test environments. Or to say explicitly that it’s 
OK if they do, I guess. 

I would also like to hear some from the distro packaging folks about what it 
would mean for us to be releasing regular version libraries frequently (expect 
at least one release every week for the entire cycle). Would you package all of 
those, or would you wait until closer to the end of the cycle and package the 
“final” versions?


>       -Sean
> -- 
> Sean Dague
> http://dague.net
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

OpenStack-dev mailing list

Reply via email to