Stefan Sperling <s...@elego.de> writes: > On Mon, Feb 21, 2011 at 01:44:35PM +0530, Noorul Islam K M wrote: >> >> This patch reduces checkout by around 23 times. > > On my system the difference is 43 seconds vs. 30 seconds.
On my low-end Linux desktop it's 7.5 seconds and 3.5 seconds, run sequentially on a SATA disk. > We lose the ability to easily spot which of the subtest is failing > if we do this. I.e. instead of: > > ... > PASS: input_validation_tests.py 19: non-working copy paths for 'status' > FAIL: input_validation_tests.py 20: non-working copy paths for 'patch' > PASS: input_validation_tests.py 21: non-working copy paths for 'switch' > ... > > all we'd get is: > > FAIL: input_validation_tests.py 1: inavlid wc and url targets When a test fails the first thing I do is look in tests.log, that will still work just as well with the combined test. I suppose we do lose out if there are multiple independent bugs, as the test will stop at the first one and not report the others. I feel we should be optimising for the common case where the tests PASS, and that optimising for FAILs is the wrong approach. I agree that combining big, complex tests, like merge, is a bad idea. But for relatively trivial tests I think reduced runtime is more important. -- Philip
