Hey Stéphane, Stéphane Graber [2014-04-02 16:19 -0400]: > LXC upstream does full automated test runs on amd64, i386 and armhf for > every commit to both the master and stable-1.0 branch. We also do test > builds and test runs using clang, push daily builds to coverity for > static code analysis and do automated cross-builds for Android. > > The upstream Jenkins server is at: https://jenkins.linuxcontainers.org
I poked around and I can only find source builds for LXC itself, and the daily download builds from lxc-templates. Where are the actual test suite runs for CI? I had a look at the test suite[1]. It covers quite a lot of the API, but only the most common cases and often only ensures that the calls don't fail: E. g. it doesn't check that a call like add_device_node() actually does what it promises. But while it doesn't cover many corner cases it certainly covers all the major workflows and functionality, especially the integration tests like "lxc-test-ubuntu". These also run in the autopkgtest which we intend to keep running for trusty after the release. I did a spot check of about 10 recent commits, and none of them were accompanied by test updates (in particular tricky stuff like https://github.com/lxc/lxc/commit/0d9acb9 which applies to relatively subtle changes which can easily regress without noticing quickly). Do you have plans to also cover testing of the templates? Bugs like https://bugs.debian.org/743425 are a bit of a bummer to have if they hit a stable release (apparently that's not what happened in that case, though, it's just something which I ran into recently). > Seeing the very large amount of SRUs we've been doing in the past > few releases, it's my belief that an MRE will save a lot of time to > Serge and I as well as the SRU team by simplifying the amount of > documentation required for our updates. I can't say that the existing test coverage would be sufficiently reliable to use it as fully automated SRU testing, but for each new microrelease we should run a test plan for the most important scenarios. But with that I have no general objection, especially as the changes you intend to do to the 1.0 branch are bug fix only (contrary to other MREs). So I don't see verifying individual bugs for SRUs as the most important issue here, but regression testing. When doing that, and limiting this MRE to bug fixes only, I'm fine with the MRE. Thanks, Martin [1] Small chuckle: createtest.c creates a busybox container and says "trusty container". Totally not important for this mail, of course :) -- Martin Pitt | http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
signature.asc
Description: Digital signature
-- technical-board mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/technical-board
