We've been working on navigating through from an interesting situation
over the past few months, but there isn't a top-level overview of what's
going on with it. That's my bad - I've been telling AJaeger I was going
to send an email out for a while.
projects with test requirements on git repo urls of other projects
There are a bunch of projects that need, for testing purposes, to depend
on other projects. The majority are either neutron or horizon plugins,
but conceptually there is nothing neutron or horizon specific about the
issue. The problem they're trying to deal with is that they are a plugin
to a service and they need to be able to import code from the service
they are a plugin to in their unit tests.
To make things even more complicated, some of the plugins actually
duepend on each other for real, not just as a "we need this for testing"
There is trouble in paradise though - which is that we don't allow git
urls in requirements files. To work around this, the projects in
question added additional pip install lines to a tox_install.sh script -
essentially bypassing the global-requirements process and system
This went unnoticed in a general sense until we started working through
removing the use of zuul-cloner which is not needed any longer in Zuul v3.
There are a few different options, but it's important to keep in mind
that we ultimately want all of the following:
* The code works
* Tests can run properly in CI
* "Depends-On" works in CI so that you can test changes cross-repo
* Tests can run properly locally for developers
* Deployment requirements are accurately communicated to deployers
The approach so far
The approach so far has been releasing service projects to PyPI and
reworking the projects to depend on those releases.
This approach takes advantage of the tox-siblings feature in the gate to
ensure we're cross-testing master of projects with each other.
There is a feature in the Zuul tox jobs we refer to as "tox-siblings"
(this is because historically - wow we have historical context for zuul
v3 now - it was implemented as a separate role) What it does is ensure
that if you are running a tox job and you add additional projects to
required-projects in the job config, that the git versions of those
projects will be installed into the tox virtualenv - but only for
projects that would have been installed by tox otherwise. This way
required-projects is both safe to use and has the effect you'd expect.
tox-siblings is intended to enable ADDITIONALLY cross-testing projects
that otherwise have a normal dependency relationship in the gate. People
have been adding jobs like cross-something-something or something-tips
in an ad-hoc manner for a while - and in many cases the git parts of
that were actually somewhat not correct - so this is an attempt to
provide the thing people want in those scenarios in a consistent manner.
But it always should be helper logic for more complex gate jobs, not as
a de-facto part of a project's basic install.
Current Approach is wrong
Unfortunately, as part of trying to unwind the plugins situation, we've
walked ourselves into a situation where the gate is the only thing that
has the correct installation information for some projects, and that's
From a networking plugin approach the "depend on release and use
tox-siblings" assumes that 'depend on release of neutron' is or can be
the common case with the ability to add a second tox job to check master
If that's not a real thing, then depending on releases + tox_siblings in
the gate is solving the wrong problem.
As there are a few different scenarios, I want to suggest we do a few
* Prefer interface libraries on PyPI that projects depend on
Like python-openstackclient and osc-lib, this is the *best* approach
for projects with plugins. Such interface libraries need to be able to
do intermediate releases - and those intermediate releases need to not
break the released version of the projects. This is the hardest and
longest thing to do as well, so it's most likely to be a multi-cycle effort.
* Treat inter-plugin depends as normal library depends
If networking-bgpvpn depends on networking-bagpipe and networking-odl,
then networking-bagpipe and networking-odl need to be released to PyPI
just like any other library in OpenStack. These are real runtime
Yes, this is more coordination work, but it's work we do everywhere
already and it's important.
If we do that for inter-plugin depends, then the normal tox jobs should
test against the most recent release of the other plugin, and people can
make a -tips style job like the openstackclient-tox-py35-tips job to
ALSO test that networking-bgpvpn works with tip of networking-odl.
* Relax our rules about git repos in test-requirements.txt
Introduce a whitelist of git repo urls, starting with:
For the service projects that have plugins that need to test against the
service they're intending to be used with in a real installation. For
those plugin projects, actually put the git urls into
test-requirements.txt. This will make the gate work AND local
development work for the scenarios where the thing that is actually
needed is always testing against tip of a corresponding service.
* In the zuul jobs, add something similar to tox-siblings but before the
initial install that will detect a git url that matches a locally
checked out repo and will swap the local copy instead so that we don't
have tox cloning directly in gate jobs.
At this point, horizon and neutron plugin projects should be able to use
normal tox jobs WITHOUT needing to list anything other than horizon and
neutron themselves in required-projects, and they can also add
project-specific -tips jobs that will add intra-plugin depends to their
required-projects so that they can test both sides of the coin.
Finally, and this is a thing we need broadly for OpenStack and not just
* Extract the tox-siblings logic into a standalone tool that can be
installed and used from tox so that it's possible to replicate a -tips
job locally. I've got this pretty much done and just need to get it
finished up. As soon as it exists I'll update python-openstackclient's
tox.ini file to use it - and people can cargo cult from there and/or we
can work it up into a documented recipe for people.
There is one more scenario or concern, which is that for the horizon
plugins, without horizon in the requirements.txt file, we can be
erroneously communicating to a deployer that they can be used standlone
without horizon. For now I think we're going to have to solve that with
a documentation note coupled with having the horizon repo link in the
test requirements ... but it might be worth pondering what we could do
to make this better. Perhaps for horizon because of that use-case we
really should be modelling horizon as cycle-plus-intermediary and should
make horizon plugins depend on horizon releases? I'm don't know that I
know the full ramifications of making that choice - so for now I think
the above approach (horizon git url in test-requiremnts) plus
documentation is safer and gives us time to consider whether all the
horizon plugin projects listing horizon in their requirements.txt is
better or worse?
OpenStack Development Mailing List (not for usage questions)