Hello all, TL;DR: This is mostly of interest for people looking at/maintaining autopkgtests and update-excuses.html or retrying failed tests.
With our current approach to running autopkgtests it sometimes happens that a package P in -proposed gets stuck because one of its dependencies D also gets uploaded and is broken. Right now we always run all tests against the entirety of -proposed, which will cause P's tests to fail due to the broken D-proposed. However, we can do better than that: When evaluating P, we should run P-proposed packages against D-release if possible, so that a broken D-proposed can't cause P-proposed to be held back. Conversely, for evaluating D-proposed all tests should be run with P-release and the binaries of D-proposed. This was first discussed a while ago on ubuntu-release [1], and more recently as a bug report [2]. Now I finished the necessary implementation in autopkgtest, britney, and the cloud worker and rolled it out. This is essentially done by telling apt via pinning to only install the binaries from one source that we want to update (called the "trigger") from -proposed, and prefer everything else from -release. An example run can be seen for the trigger pyflakes [3], where its reverse dependency ubuntu-dev-tools now ran [4] with python-pyflakes 1.0.0-3 from -proposed (as that is the trigger), but e. g. python-gi 3.18.0-1 from -release (although 3.18.2-2 is in -proposed). So if the new pygobject 3.18.2-2 were broken, we could now still promote pyflakes. Note that apt pinning is rather limited. Unfortunately you can't say "from -proposed use these packages and any of their dependencies which are not satisfiable in -release" -- pinning is not clever enough for that. This means that the tests for e. g. gtk+3.0 where the -proposed version depends on another package in -proposed cannot be run in this isolated manner, and autopkgtest will fall back to globally enabling -proposed. You will see something like adt-run: WARNING: Packages src:gtk+3.0 are not installable with per-package pinning in proposed, falling back to using all packages from proposed: in the log, e. g. in [5]. Ideas how to improve that are highly appreciated! This comes at the expense of having to do more test runs, as one particular test run is now specific to a particular trigger. This also means that the notion of "please re-run test foo" is now meaningless, you always have to re-run a test against a given trigger. (Note that this was already the case for kernel packages, but now we use this approach for everything). In a test log, you can see the test trigger at the top as e. g. ADT_TEST_TRIGGERS=pyflakes/1.0.0-3 result.tar's "testinfo.json" contains the trigger in machine readable form. I plan to expose this in a more friendly manner on http://autopkgtest.ubuntu.com [6] soon. Martin [1] https://lists.ubuntu.com/archives/ubuntu-release/2015-August/003345.html [2] https://launchpad.net/bugs/1503150 [3] http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#pyflakes [4] https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/i386/u/ubuntuone-dev-tools/20151110_123612@/log.gz [5] https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/i386/libr/libreoffice/20151110_131142@/log.gz [6] https://bugs.launchpad.net/auto-package-testing/+bug/1511780 -- Martin Pitt | http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
signature.asc
Description: Digital signature
-- ubuntu-devel mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
