Steve Langasek [2014-12-10 0:57 -0800]: > Well, except that still doesn't help us at all, because the specific tests > we're trying to implement here are system-image upgrade tests - which means > we need to have control over the base version of OS that's installed on the > device. And this has been specifically called out as something that's not > supported.
Right. I discussed this with Barry a few weeks ago. In the "old" Jenkins world, these would be custom Jenkins jobs which install the desired old image version, and then call the test. TTBOMK the current UCI airline does not support any of that yet, it only offers pre-allocated cloud instances. Setting up the testbed can happen totally outside of adt-run, or we capture that logic into an ssh setup script (see man adt-virt-ssh) so that one has a single adt-run command to call. But in either way, the infrastructure you run this on obviously needs to support this kind of operation, i. e. that at some level something can call ubuntu-device-flash. > So I think that still leaves us at square one regarding automation of the > tests. Provisioning the devices is the time-consuming part anyway; wrapping > a call to 'system-image-cli' on the phone inside a call to 'adt-run', when a > QA person is still going to have to manually flash the device between each > individual test, isn't really an improvement. Right, I don't think the concept of "test this package" maps very well to this use case. It's useful for complex upgrade tests like we do on the desktop, where you want to run post-upgrade tests, but not if the only thing you need to call is system-image-cli, then your test control will essentially only have that one command. > How do we automate the setup of the test bed? If we assume that's always > external to DEP8, then that just means I need some other interface to write > to that isn't DEP8. In theory you could put those into a test run by adt-run, and then pipe this through the null runner and require that you have an adb device connected. I'm not sure whether this is any more elegant or easier to support (my gut feeling is that it isn't). Martin -- Martin Pitt | http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
signature.asc
Description: Digital signature
-- Mailing list: https://launchpad.net/~canonical-ci-engineering Post to : [email protected] Unsubscribe : https://launchpad.net/~canonical-ci-engineering More help : https://help.launchpad.net/ListHelp
