Over the past week Monty and I have been running at how to solve the
problem that because of the way we install devstack nodes, our loose
coordination of projects causes some "interesting" issues.
===============
The basic problem
===============
We install projects in the gate in this order (based on linear order in
devstack):
install_infra
install_oslo
install_keystoneclient
install_glanceclient
install_cinderclient
install_novaclient
(then conditionally if certain services are enabled)
install_swiftclient
install_neutronclient
install_heatclient
install_keystone
install_swift
install_glance
install_cinder
install_neutron
install_neutron_third_party
install_nova
install_horizon
install_ceilometerclient
install_ceilometer
install_heat
install_tempest
At every point we "python setup.py develop" to pull in requirements.
That means, that if in install_glance we require "jsonschema" we'll
install the latest, i.e. 2.0. And if at install_tempest we require
"jsonschema>=1.3.0,!=1.4.0,<2" (which is the global requirements line),
we will uninstall jsonschema 2.0. And if something, like cinder, dragged
in something that required jsonschema, and uses entry points, which
really care about python package versions, it will be built against 2.0,
which will have been removed, and we'll have 1.3 instead.
Note: that's not a theoretical scenario, that's why no one could merge
code last weekend.
We had an equivalent wedge with horizon a month ago, which was the
python-*client uncap push.
Basically.... we're spinning 20 plates all at the same time, which all
have different review times, +2 teams, and entropy means they tend to
drift away from each other over time. It took us a month to get
python-*clients uncapped across all the projects because of the
intricate ways in which our projects require our other projects in
interesting ways.
================
Proposed Solution - part #1 - use global requirements in devstack gate
================
The basic idea is simple, in the devstack gate, we're going to forcibly
update all the project's requirements.txt files to global requirements
versions, before running setup.py. That means that any project which
requires a thing will require it at exactly the same version range as
everyone else. Which ensures that we don't have this uninstall /
reinstall issue.
We are really close to having this working. Monty and I have been
running at it all week, and I actually think this requirements review -
https://review.openstack.org/#/c/39461/ might be the last bit we need
before we can land the devstack changes for this.
Things this solves:
* makes global requirements actually mean something, right now the
sqlalchemy line is known wrong, which prevents nova from working. There
are probably other issues as well.
* ensures we really have a set of requirements that work for all projects.
Things this will allow:
* devstack gating on proposed changes to global-requirements, so
you'll know if all the projects can work with global-requirements before
you land them.
Things that we had to change along the way:
* oslo.config / oslo.messaging now in devstack - in order to let
projects use prerelease features of those projects, in this model, we
needed their git trees to be in our environment, like the python* clients.
* oslo.config in the devstack-gate (landing soon) - we'll now know on
every oslo.config proposed commit that it works with all the projects in
the devstack gate. Once oslo.messaging is proposed for use, we'll
devstack-gate that as well.
Assuming sqlalchemy is the only requirements issue (it's likely that is
the case), the rest of the gating around this should land today. This
should just slide in quietly, and not cause any problems, however, there
is an interesting caveat....
=================
Interesting Caveat
=================
Global requirements will be used by everyone in the devstack-gate and
the grenade-gate, *however* in unit tests, projects will still be using
their local requirements. These might not be the same (actually, for
almost all projects I can tell you these aren't the same today).
We need to think about the resolution here. There are a number of ones I
can think of, but this is going to clearly require more thought to get
right. We need to be able to support things like prerelease Oslo, for
instance. So lets just say this is tabled until phase #1 is fixed.
-Sean
--
Sean Dague
http://dague.net
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
