Should we add random sleeps to DRT? It'll certainly help find some regressions (and even security bugs). Of course the down-side is that it makes tests non-repeatable and difficult to reason about.
I'm baffled by your priorities and don't know how to continue this conversation productively. Sorry. Cheers, -a On Fri, Oct 26, 2012 at 12:43 PM, Alexey Proskuryakov <[email protected]> wrote: > > 26.10.2012, в 11:04, Antti Koivisto <[email protected]> написал(а): > > The reality is that this "test coverage" today shows up as flakiness and >> so is ignored anyway, meaning we don't actually have useful coverage here. >> Even when flakiness is investigated, the "fix" is to cache-bust using >> unique URL params, which just means we "lose" the coverage you describe for >> that test, anyway. >> > > I think that this is the real issue here. Test flakiness is very important > to investigate, this often leads to discovery of bad bugs, including > security ones. The phrase "flaky test" often misplaces the blame. > > When making cache related changes I have frequently found bugs from my > patches because some seemingly random test started failing and I > investigated. Without the test coverage some of those bugs would probably > now be in the tree. > > > I agree with Antti. Finding regressions is what tests are for, and it > would be difficult to make enough explicit tests to compensate for such > loss of coverage. It would certainly be very unfortunate to lose test > coverage without even an attempt to compensate for that. > > > - WBR, Alexey Proskuryakov > > > > _______________________________________________ > webkit-dev mailing list > [email protected] > http://lists.webkit.org/mailman/listinfo/webkit-dev > >
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo/webkit-dev
