Ricardo Salveti <[email protected]> writes: >>> The first idea I had was to basically have a way at lava-tests to skip >>> the step which it installs the test dependencies. At least with that >>> approach, we could already have the dependencies in place for the test >>> cases we care about. >> >> I think I'm coming around to the android team's POV here: tests don't >> have dependencies -- you test things in the base system (for a while we >> were running the ubuntu-leb-graphics tests on a nano image, which >> installed X and everything else first -- madness!). > > It can have, but if they are part of the image itself, we can at least > cover that by default.
I don't quite understand here, I'm afraid. > The problem is that supporting more test cases is directly connected > on producing new kinds of images (with extra packages and such), which > can be painful. Maybe we should make that easier then? >>> Otherwise, we'd need a way to dynamically install the dependencies, >>> which can be quite complicated with OE as we'd need to create and >>> export a repository. >>> >>>> One of the things I've raised in the past that we should look at is >>>> consolidation of lava-test and lava-android-test by means of a host driven >>>> test framework, and this is one reason why. >>> >>> Can you explain more about how this host driven test framework would work? >> >> So the idea is that "test installation" arranges for the test to run on >> next boot of the test system, putting the result output in a known >> location. Then we boot the device, wait for the test to complete (hand >> waving here), fetch the results from the known location and parse them >> into a bundle. There will be a system dependent part around how the >> tests are started -- for ubuntu you might create an upstart job, for OE >> I guess you'll do something else -- but that feels like a small amount >> of variation. >> >> How you get the result output after the tests have run will also vary by >> deployment approach, but we already handle this today to get the bundles >> off the device. > > Sounds fine, while you don't have total progress in real time, it's a > lot simpler to follow up the execution and parsing. We could spam the output to the serial console as well while the test is running -- probably a good idea, in fact. > While it can help depending on the environment you have, it'll not > improve the current situation a lot (it'll probably just make the > maintenance a bit easier). One of the nice things is that it means that we don't have to install (and have working) things like lava-test on the DUT, which is nice for things like fast models of new architectures :-) >>>> Certainly the dispatcher will have to support another means of >>>> installation, >>>> just like we have to for Android images. If linaro-media-create is used, >>>> then this makes things slightly better, but I'm sure it'll need at least a >>>> bit of special handling in the dispatcher, though probably not as bad as >>>> android was. I don't foresee a big problem here. >>> >>> I believe we could even use the hwpacks we produce for Ubuntu here, >>> but only dealing with the kernel and bootloader (skipping all other >>> packages). As we don't care much about upgradability, we could have a >>> logic at linaro-media-create to just extract the debian packages >>> related with the kernel and bootloader (something that happens in some >>> way anyway). >> >> I guess. I don't think the way it happens today is at all reusable >> fwiw: it looks like it looks it depends on linaro-hwpack-install >> installing the a package that creates a path such as >> boot/vmlinuz-*-linaro-omap. I presume it wouldn't be very hard to make >> a guess at which deb creates this file and use ar/tar to "install" it, >> but it would be new code I think. >> >>> With that logic in place, we could support any rootfs, even Android >>> (and share the same basic set of hwpacks we also use at Ubuntu). > > The code for such usage is mostly done at > https://code.launchpad.net/~rsalveti/linaro-image-tools/generic-oe-support/, > and I was able to test with the current OE images I produced locally. > I'll be testing it properly in different use cases in the next > following days, but should be ready to merge/review soon. Ah, cool. > The idea is basically to use linaro-media-create the same way as it's > done with Ubuntu's images, so we avoid changing much on the interface > of LAVA or any other tool that relies on lmc. > > The only problem I see in the future, which will require a bit of > re-work again (on lmc), is when we start bootstrapping a new > architecture, as we'll not have any qemu support to run native code > (post installs or similar). Yeah, that's another reason to do blackbox-style/host-side testing... > While it's not that complicated, it can get messy because of the > current way lmc is flashing up the images. Cheers, mwh _______________________________________________ linaro-validation mailing list [email protected] http://lists.linaro.org/mailman/listinfo/linaro-validation
