On 29 Apr 2013, at 06:55, Frank Loeffler <[email protected]> wrote: > On Sun, Apr 28, 2013 at 11:58:00PM -0400, Erik Schnetter wrote: >> This is not really news, given that all test cases are run automatically >> after every commit: <https://trac.einsteintoolkit.org/>; look for "Status". > > You are right. This has been steady for some months now, in no small > part due to these automated tests - thanks to Barry and Ian. I was > referring to a more distant past, when it was a bit like this: > > http://geek-and-poke.com/2011/08/hudson-status-monitor.html
Without automated testing, it used to be the case that the full set of tests would likely not be run between two releases. So one had to fix all the problems at once, and there was no indication of which commit caused the failure. Over the past year, as we have brought the automated testing system online, we have usually flagged these issues instantly, and it is usually very clear which commit has broken the tests. I'm not sure I agree with Erik's statement that we have had more problems with the repositories and automated test system than with the code. People have frequently committed code which broke some of the tests (which usually results in a private email from me); it's just that the code problems have been fixed very quickly after being discovered as it was clear what caused them. Ideally, the tests would be easy enough to run that everyone would run them before committing, but this is not the case at the moment, as they take too long. If we consider the automated test system stable enough now, we can switch to a mode where the test failures are reported to a mailing list (maybe the commits mailing list, or a new one) and the original committer, so that everyone is aware of the problems who wants to be. -- Ian Hinder http://numrel.aei.mpg.de/people/hinder
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Users mailing list [email protected] http://lists.einsteintoolkit.org/mailman/listinfo/users
