There hasn't been a Python 2 release in 8 years, and during that time
we've gotten used to the idea that that's the way things go. However,
with the switch to Python 3 looming (we will drop support for Python 2
in the U release[1]), history is no longer a good guide: Python 3
releases drop as often as every year. We are already feeling the pain
from this, as Linux distros have largely already completed the shift to
Python 3, and those that have are on versions newer than the py35 we
currently have in gate jobs.
We have traditionally held to the principle that we want each release to
support the latest release of CentOS and the latest LTS release of
Ubuntu, as they existed at the beginning of the release cycle.[2]
Currently this means in practice one version of py2 and one of py3, but
in the future it will mean two, usually different, versions of py3.
There are two separate issues that we need to address: unit tests (we'll
define this as code tested in isolation, within or spawned from within
the testing process), and integration tests (we'll define this as code
running in its own process, tested from the outside). I have two
separate but related proposal for how to handle those.
I'd like to avoid discussion which versions of things we think should be
supported in Stein in this thread. Let's come up with a process that we
think is a good one to take into T and beyond, and then retroactively
apply it to Stein. Competing proposals are of course welcome, in
addition to feedback on this one.
Unit Tests
----------
For unit tests, the most important thing is to test on the versions of
Python we target. It's less important to be using the exact distro that
we want to target, because unit tests generally won't interact with
stuff outside of Python.
I'd like to propose that we handle this by setting up a unit test
template in openstack-zuul-jobs for each release. So for Stein we'd have
openstack-python3-stein-jobs. This template would contain:
* A voting gate job for the highest minor version of py3 we want to
support in that release.
* A voting gate job for the lowest minor version of py3 we want to
support in that release.
* A periodic job for any interim minor releases.
* (Starting late in the cycle) a non-voting check job for the highest
minor version of py3 we want to support in the *next* release (if
different), on the master branch only.
So, for example, (and this is still under active debate) for Stein we
might have gating jobs for py35 and py37, with a periodic job for py36.
The T jobs might only have voting py36 and py37 jobs, but late in the T
cycle we might add a non-voting py38 job on master so that people who
haven't switched to the U template yet can see what, if anything,
they'll need to fix.
We'll run the unit tests on any distro we can find that supports the
version of Python we want. It could be a non-LTS Ubuntu, Fedora, Debian
unstable, whatever it takes. We won't wait for an LTS Ubuntu to have a
particular Python version before trying to test it.
Before the start of each cycle, the TC would determine which range of
versions we want to support, on the basis of the latest one we can find
in any distro and the earliest one we're likely to need in one of the
supported Linux distros. There will be a project-wide goal to switch the
testing template from e.g. openstack-python3-stein-jobs to
openstack-python3-treasure-jobs for every repo before the end of the
cycle. We'll have goal champions as usual following up and helping teams
with the process. We'll know where the problem areas are because we'll
have added non-voting jobs for any new Python versions to the previous
release's template.
Integration Tests
-----------------
Integration tests do test, amongst other things, integration with
non-openstack-supplied things in the distro, so it's important that we
test on the actual distros we have identified as popular.[2] It's also
important that every project be testing on the same distro at the end of
a release, so we can be sure they all work together for users.
When a new release of CentOS or a new LTS release of Ubuntu comes out,
the TC will create a project-wide goal for the *next* release cycle to
switch all integration tests over to that distro. It's up to individual
projects to make the switch for the tests that they own (e.g. it'd be
the QA team for Tempest, but other individual projects for their own
jobs). Again, there'll be a goal champion to monitor and follow up.
[1]
https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html
[2]
https://governance.openstack.org/tc/reference/project-testing-interface.html#linux-distributions
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
