On Nov 18, 2015, at 03:03 PM, Colin Watson wrote: > * projects using buildout (Launchpad itself; lazr.* and friends)
Note that several of the lazr packages that I've touched recently-ish (e.g. smtptest[*], config, delegates) I've already de-buildout-ified and converted to tox. tox is not only a great multi-version test regime, but the virtualenvs it builds are perfectly suitable for all the normal venv-y stuff you might want to do while developing a library. You can also relegate the test-requirements (i.e. non-runtime reqs) to the tox.ini so as to keep the setup.py fairly clean for better integration with distro tools like pybuild. I recommend we standardize on tox. > Tried and tested. OTOH it's also a massive pain. We're stuck on an > old version at least for Launchpad itself because we're relying on > buildout's partial isolation support which was removed in buildout 2; > and buildout doesn't support setup_requires, which many of our > dependencies now make use of directly or transitively, particularly > via pbr. It is massively unclear how to run tests at all on many > lazr.* packages, never mind how to do so if you happen to be > attempting a Python 3 port; figuring out the exact details of how to > run ./bootstrap.py etc. is awkward at best and we have quite a lot of > bitrot here. Yes, let's get rid of buildout. > * projects using virtualenv and pip (turnip, txpkgupload, rutabaga) > > Relatively new to us, but is mostly working well. We have some > problems with coherent requirements management, and different > projects use requirements vs. constraints differently, which I'd like > to fix before we extend this to more things. +1, and see above. >Aesthetically I think I prefer declaring them in setup.py and using "-e ." in >requirements.txt, since we generally seem to need editable mode anyway; but >the pbr approach would work too, so I don't feel very strongly about this. I've only recently seen a few knock-on problems related to pbr, but I don't remember the details (Robert probably does) and I think they've since been fixed. I actually didn't know about the `-e .` trick until I read Donald's post... neat! > * All projects must use a virtual environment and pip to install their > dependencies. (This is relatively straightforward for most projects; > Launchpad itself will take a while.) Via tox, where appropriate. An added benefit is that you generally don't need to manually setup and manage venvs. > * All projects must declare their direct dependencies (only - must not be > flattened/resolved) in setup.py and use "-e ." in requirements.txt, to be > used with "pip install -r requirements.txt". (Alternative: declare them > in requirements.txt and use pbr's requirements parsing support.) +1 for setup.py and `-e .`. I don't have enough experience with pbr to know whether it brings enough to the table to offset an additional dependency. > * Projects that are the top level of a deployment must include a > flattened list of dependency versions in constraints.txt, as produced > by "pip freeze", to be used with "pip install -c constraints.txt -e > .". All the necessary packages should be included in a suitable > dependencies branch. Unless they are deploying to a system that > already has pip 7.1 or newer, they will typically need a > bootstrap-requirements.txt that upgrades to modern versions of pip > and setuptools in order for constraints to be available. I've been stalled getting pip upgraded in Debian and Ubuntu, and I *really* need to get back to it. My TODO list is here: http://anonscm.debian.org/cgit/python-modules/packages/python-pip.git/tree/debian/changelog There's a fair bit of work to do before actually touching pip, and any help would be greatly appreciated! > * Any project that requires a fork of a dependency (i.e. a version not > on PyPI) must pin the forked version in constraints.txt. If the > forked version is required for development builds, then it should > also include a suitable VCS entry in requirements.txt (see > https://pip.readthedocs.org/en/stable/reference/pip_install/#vcs-support). > Note that this is safe because deployments use constraints.txt, not > requirements.txt. > > * Testing libraries against application deployment versions should be > achieved using "pip install -c /path/to/application/constraints.txt > -e .". Unless they are themselves the top level of a deployment, > libraries should not include full pinned constraints; they will only > get out of date. > >Does this all sound sensible, or have I missed something? It all sounds very reasonable. These are old code bases so modernizing them is a good thing. Cheers, -Barry P.S. Can we switch to git? (/me hears the sound of worms wriggling in cans) [*] For those of you wanting to do some fun stuff with asyncio in Python 3, take a look at our next generation SMTP testing framework, aiosmtpd: https://gitlab.com/python-smtpd-hackers/aiosmtpd _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
