On 5 June 2012 23:34, Alexander Sack <[email protected]> wrote: > On Tue, Jun 5, 2012 at 5:26 PM, Zach Pfeffer <[email protected]> wrote: >> On 5 June 2012 18:23, Alexander Sack <[email protected]> wrote: >>> I feel stopping and rebooting and continuing with the next test is >>> what we want to aim for. >>> >>> On this front I wonder if we should directly go for rebooting for >>> _all_ tests to ensure that every test gets executed the same runtime >>> environment. >>> >>> Big benefit is obviously that tests can then stop services, change >>> runtime state etc. as much as they like without bothering about >>> bringing the system back into a clean state. >>> >>> Would that be hard? Why wouldn't we do this? >> >> Seems like a good idea in theory, but in practice it may cause testing >> to take a long time. Plus, what constitutes a test boundary? I think > > I don't think this would really extend the testing time much. Majority > of time is usually spend in flashing the image. The booting and > running should be not of a big time sink; the little bit we loose we > get back from keeping the suite running "isolated". > >> if we do the fail then restart then we get the best of both worlds, >> we're able to run multiple tests and restart if we get the system into >> a really weird state. > > I would think "one suite" is a good test boundary (basically the names > you currently put in TEST_PLAN).
Sure. I actually okay with this. If we do this though, we may want to change things so that each test is a tuple with a test name and a timeout for that test. > > > -- > Alexander Sack > Technical Director, Linaro Platform Teams > http://www.linaro.org | Open source software for ARM SoCs > http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog -- Zach Pfeffer Android Platform Team Lead, Linaro Platform Teams Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog _______________________________________________ linaro-validation mailing list [email protected] http://lists.linaro.org/mailman/listinfo/linaro-validation
