Hi everybody,

We've been working on navigating through from an interesting situation over the past few months, but there isn't a top-level overview of what's going on with it. That's my bad - I've been telling AJaeger I was going to send an email out for a while.

projects with test requirements on git repo urls of other projects
------------------------------------------------------------------

There are a bunch of projects that need, for testing purposes, to depend on other projects. The majority are either neutron or horizon plugins, but conceptually there is nothing neutron or horizon specific about the issue. The problem they're trying to deal with is that they are a plugin to a service and they need to be able to import code from the service they are a plugin to in their unit tests.

To make things even more complicated, some of the plugins actually duepend on each other for real, not just as a "we need this for testing"

There is trouble in paradise though - which is that we don't allow git urls in requirements files. To work around this, the projects in question added additional pip install lines to a tox_install.sh script - essentially bypassing the global-requirements process and system completely.

This went unnoticed in a general sense until we started working through removing the use of zuul-cloner which is not needed any longer in Zuul v3.

unwinding things
----------------

There are a few different options, but it's important to keep in mind that we ultimately want all of the following:

* The code works
* Tests can run properly in CI
* "Depends-On" works in CI so that you can test changes cross-repo
* Tests can run properly locally for developers
* Deployment requirements are accurately communicated to deployers

The approach so far
-------------------

The approach so far has been releasing service projects to PyPI and reworking the projects to depend on those releases.

This approach takes advantage of the tox-siblings feature in the gate to ensure we're cross-testing master of projects with each other.

tox-siblings
-----------

There is a feature in the Zuul tox jobs we refer to as "tox-siblings" (this is because historically - wow we have historical context for zuul v3 now - it was implemented as a separate role) What it does is ensure that if you are running a tox job and you add additional projects to required-projects in the job config, that the git versions of those projects will be installed into the tox virtualenv - but only for projects that would have been installed by tox otherwise. This way required-projects is both safe to use and has the effect you'd expect.

tox-siblings is intended to enable ADDITIONALLY cross-testing projects that otherwise have a normal dependency relationship in the gate. People have been adding jobs like cross-something-something or something-tips in an ad-hoc manner for a while - and in many cases the git parts of that were actually somewhat not correct - so this is an attempt to provide the thing people want in those scenarios in a consistent manner. But it always should be helper logic for more complex gate jobs, not as a de-facto part of a project's basic install.

Current Approach is wrong
------------------------

Unfortunately, as part of trying to unwind the plugins situation, we've walked ourselves into a situation where the gate is the only thing that has the correct installation information for some projects, and that's not good.

From a networking plugin approach the "depend on release and use tox-siblings" assumes that 'depend on release of neutron' is or can be the common case with the ability to add a second tox job to check master against master.

If that's not a real thing, then depending on releases + tox_siblings in the gate is solving the wrong problem.

Specific Suggestions
--------------------

As there are a few different scenarios, I want to suggest we do a few different things.

* Prefer interface libraries on PyPI that projects depend on

Like python-openstackclient and osc-lib, this is the *best* approach
for projects with plugins. Such interface libraries need to be able to do intermediate releases - and those intermediate releases need to not break the released version of the projects. This is the hardest and longest thing to do as well, so it's most likely to be a multi-cycle effort.

* Treat inter-plugin depends as normal library depends

If networking-bgpvpn depends on networking-bagpipe and networking-odl, then networking-bagpipe and networking-odl need to be released to PyPI just like any other library in OpenStack. These are real runtime dependencies.

Yes, this is more coordination work, but it's work we do everywhere already and it's important.

If we do that for inter-plugin depends, then the normal tox jobs should test against the most recent release of the other plugin, and people can make a -tips style job like the openstackclient-tox-py35-tips job to ALSO test that networking-bgpvpn works with tip of networking-odl.

* Relax our rules about git repos in test-requirements.txt

Introduce a whitelist of git repo urls, starting with:

  * https://git.openstack.org/openstack/neutron
  * https://git.openstack.org/openstack/horizon

For the service projects that have plugins that need to test against the
service they're intending to be used with in a real installation. For those plugin projects, actually put the git urls into test-requirements.txt. This will make the gate work AND local development work for the scenarios where the thing that is actually needed is always testing against tip of a corresponding service.

* In the zuul jobs, add something similar to tox-siblings but before the initial install that will detect a git url that matches a locally checked out repo and will swap the local copy instead so that we don't have tox cloning directly in gate jobs.

At this point, horizon and neutron plugin projects should be able to use normal tox jobs WITHOUT needing to list anything other than horizon and neutron themselves in required-projects, and they can also add project-specific -tips jobs that will add intra-plugin depends to their required-projects so that they can test both sides of the coin.

Finally, and this is a thing we need broadly for OpenStack and not just neutron/horizon plugins:

* Extract the tox-siblings logic into a standalone tool that can be installed and used from tox so that it's possible to replicate a -tips job locally. I've got this pretty much done and just need to get it finished up. As soon as it exists I'll update python-openstackclient's tox.ini file to use it - and people can cargo cult from there and/or we can work it up into a documented recipe for people.

There is one more scenario or concern, which is that for the horizon plugins, without horizon in the requirements.txt file, we can be erroneously communicating to a deployer that they can be used standlone without horizon. For now I think we're going to have to solve that with a documentation note coupled with having the horizon repo link in the test requirements ... but it might be worth pondering what we could do to make this better. Perhaps for horizon because of that use-case we really should be modelling horizon as cycle-plus-intermediary and should make horizon plugins depend on horizon releases? I'm don't know that I know the full ramifications of making that choice - so for now I think the above approach (horizon git url in test-requiremnts) plus documentation is safer and gives us time to consider whether all the horizon plugin projects listing horizon in their requirements.txt is better or worse?

Thoughts?

Monty

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to