On 12 June 2014 18:21, Patrick Ohly <[email protected]> wrote: > On Thu, 2014-06-12 at 18:06 +0300, Stoppa, Igor wrote:
[...] >> 3) if said images can be built successfully, they are flashed and >> tested (in a remote testing farm, which is part of the building >> infrastructure). >> Currently this happens with a fixed set of test cases, but it is >> possible to make it conditional. For example to what comes from the >> .spec file(s) of the package(s) that are new in the specific build. > > Or that were recompiled because one of their dependencies changed. This > would help detecting when a library update breaks other packages. This > is very hard to test otherwise, both for the library maintainer and the > maintainer of the component depending on a library. yes, but in these cases I would rather favor a different approach: having automated testing allows to do a lot of it :-) so the goal is to keep the execution of the individual test case fairly short Therefore I would prefer to have some relatively complex (meaning exerting a large portion of he SW architecture) canary bird test case and rely on the fact that we can run it often. The goal is to identify the largest possible set of breakages and stop the change that introduces them. If we can run it often enough, we can have small enough changes that it's trivial to spot the cause of the regression(s). [...] > One reason for using a remote farm is that developers do not necessarily > have access to real test devices. Using the same hardware for different > developers is more cost-efficient, too, in particular when those > developers are in different time zones and not active at the same time. sure, it's just that the HW needs to come from somewhere :-D both the IVIs or whatever we want to use and the servers that will create the images. One more reason to search for consensus also beyond the circle of developers ;-) -- cheers, igor _______________________________________________ Dev mailing list [email protected] https://lists.tizen.org/listinfo/dev
