On Sat, Apr 14, 2012 at 3:37 AM, Jonathan Lange <j...@mumak.net> wrote: Bah, I forgot to address these.
> I'd rather raise ideas here in case because they probably need refinement: > > * reduce test suite run time without reducing coverage This would be nice. I think it's hard, short of the parallelisation style work that Gary et al are doing at the moment, to have a good developer time/result tradeoff. E.g. a week of developer time on speedups giving a 30 minute improvement(a massive one!) would still take 80 test runs to show a return. Good to do at small scale, but hardware - once we parallelise - is /relatively/ cheap. We should strive to keep it snappy, but time spent just making the existing tests faster is hard to reason about. Perhaps this: timebox any efforts in this direction. How much of an improvement is worth 1K lines? I don't know. You could say that an improvement equal to the time to run the tests for that new 1K lines would offset the 1K lines entirely : and in terms of test speed it would, but 1K lines also brings new defects, new concepts to understand, new places for bugs... I'd probably want a factor of 2 improvment to waive LoC on the basis of test run time: if you add 1K LoC with tests taking 30 seconds, I'd waive the LoC if you reduce the existing test run time by 60 seconds. Francis may have a different answer :) > * improve test suite coverage (I assume this bypasses LoC restrictions?) Not at all. Coverage is good. LoC restrictions are good. Doing both is excellent. We have a raft of slow multi-layer tests masquerading as unit tests, we test some parts of the system thousands of times more than other parts because of this. I'm sure you can have simpler smaller targeted tests with better coverage in less LoC. > * something about making tests more focused Meh. Doesn't, IME, make a significant difference to maintainability as a first order effect. Second order effect sure - but theres a direct driver for those (such as 'slow tests' or 'too much LoC'.). -Rob _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
