There seems to be agreement that this is broadly a good direction to pursue, so I proposed a TC resolution. Let's shift discussion to the review:


On 19/10/18 11:17 AM, Zane Bitter wrote:
There hasn't been a Python 2 release in 8 years, and during that time we've gotten used to the idea that that's the way things go. However, with the switch to Python 3 looming (we will drop support for Python 2 in the U release[1]), history is no longer a good guide: Python 3 releases drop as often as every year. We are already feeling the pain from this, as Linux distros have largely already completed the shift to Python 3, and those that have are on versions newer than the py35 we currently have in gate jobs.

We have traditionally held to the principle that we want each release to support the latest release of CentOS and the latest LTS release of Ubuntu, as they existed at the beginning of the release cycle.[2] Currently this means in practice one version of py2 and one of py3, but in the future it will mean two, usually different, versions of py3.

There are two separate issues that we need to address: unit tests (we'll define this as code tested in isolation, within or spawned from within the testing process), and integration tests (we'll define this as code running in its own process, tested from the outside). I have two separate but related proposal for how to handle those.

I'd like to avoid discussion which versions of things we think should be supported in Stein in this thread. Let's come up with a process that we think is a good one to take into T and beyond, and then retroactively apply it to Stein. Competing proposals are of course welcome, in addition to feedback on this one.

Unit Tests

For unit tests, the most important thing is to test on the versions of Python we target. It's less important to be using the exact distro that we want to target, because unit tests generally won't interact with stuff outside of Python.

I'd like to propose that we handle this by setting up a unit test template in openstack-zuul-jobs for each release. So for Stein we'd have openstack-python3-stein-jobs. This template would contain:

* A voting gate job for the highest minor version of py3 we want to support in that release. * A voting gate job for the lowest minor version of py3 we want to support in that release.
* A periodic job for any interim minor releases.
* (Starting late in the cycle) a non-voting check job for the highest minor version of py3 we want to support in the *next* release (if different), on the master branch only.

So, for example, (and this is still under active debate) for Stein we might have gating jobs for py35 and py37, with a periodic job for py36. The T jobs might only have voting py36 and py37 jobs, but late in the T cycle we might add a non-voting py38 job on master so that people who haven't switched to the U template yet can see what, if anything, they'll need to fix.

We'll run the unit tests on any distro we can find that supports the version of Python we want. It could be a non-LTS Ubuntu, Fedora, Debian unstable, whatever it takes. We won't wait for an LTS Ubuntu to have a particular Python version before trying to test it.

Before the start of each cycle, the TC would determine which range of versions we want to support, on the basis of the latest one we can find in any distro and the earliest one we're likely to need in one of the supported Linux distros. There will be a project-wide goal to switch the testing template from e.g. openstack-python3-stein-jobs to openstack-python3-treasure-jobs for every repo before the end of the cycle. We'll have goal champions as usual following up and helping teams with the process. We'll know where the problem areas are because we'll have added non-voting jobs for any new Python versions to the previous release's template.

Integration Tests

Integration tests do test, amongst other things, integration with non-openstack-supplied things in the distro, so it's important that we test on the actual distros we have identified as popular.[2] It's also important that every project be testing on the same distro at the end of a release, so we can be sure they all work together for users.

When a new release of CentOS or a new LTS release of Ubuntu comes out, the TC will create a project-wide goal for the *next* release cycle to switch all integration tests over to that distro. It's up to individual projects to make the switch for the tests that they own (e.g. it'd be the QA team for Tempest, but other individual projects for their own jobs). Again, there'll be a goal champion to monitor and follow up.

[1] [2]

OpenStack Development Mailing List (not for usage questions)

Reply via email to