Package: autopkgtest Version: 5.32 Severity: wishlist tl;dr:
autopkgtest is slower than it needs to be for packages with many tests. I can see at least one easy opportunity for a potentially substantial improvement, and have another more doubtful idea. Background and factual information: src:dgit has over a hundred autopkgtests (115 in all right now, but some don't run in CI due to Restrictions; only 108 run in CI). This is convenient for isolating failures. The tests are grouped into (currently) 26 stanzas in debian/tests/control. These tests usually take around 40 minutes to run in a formal run under autopkgtest. Sometimes the time taken exceeds the default 1-hour timeout in salsa CI. Some of the tests are very fast - one or two seconds. Some take much longer. (During a manual run outside autopkgtest, a complete run of the tests takes about 8 minutes, but that's with CPU parallelism so isn't comparable. I haven't done a non-parallel benchmark run.) Proposal 1: Observing the logs, I see that even for tests in the same stanza, autopkgtest prpeares and installs an autopkgtest-satdep package, between every test. For short tests, this dominates the execution time. In the logs eg https://salsa.debian.org/dgit-team/dgit/-/jobs/5270905 it seems to take about 6 seconds. So I think skipping the testbed re-preparation for tests within the same stanza would save about 6 seconds per such test, or 530-ish seconds for each run. Ie, it might cut 20% off the total runtime for this package. I'm hoping that this would be relatively straightforward to implement. Proposal 2: *Re*installing packages involved with many of the tests seems quite expensive. For src:dgit, most of the tests involve installing dgit.deb. dgit and its dependencies seem to take about 20s to install. If the test stanzas could be sequenced into "chains" where each test is has a monotonically longer dependency list, the intervening satdep installation, to get from one test stanza in the chain, to the nest stanza, wouldn't need to involve *reinstalling* anything. I'm not sure how much time this would save. Also it's not clear to me that this would always be 100% faithful. After dependency resolution is not monotonic: an extra dependency might result in *deinstallation* of packages. So this proposal is more fanciful but I thought I would share it anyway. Regards,c Ian. -- Ian Jackson <ijack...@chiark.greenend.org.uk> These opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
