Since we have had some bad code breakage in CVS in the recent past
(and some of it wasn't noticed / publicly mentioned for about a week),
perhaps we should institute some kind of nightly regression testing
that automatically runs at about 3am or so (cron is your friend).
I realize that we can't have a full automated install every night --
that would be really hard to do (easiest example: GUI interaction).
But we could do some basic tests that would have caught most of the
errors that have occurred over the last 3 weeks.
For example, a good starter might be something along the lines of:
1 - do a CVS checkout
- run autogen.sh
- run ./configure
- run make dist
--> NOTE: this is from the philosophy of "test what you ship; ship
what you test"
2 - un-tar the resulting tarball
- cd into the new directory
- run ./configure
- run make install
3 - cd into the installation directory
- run ./install_cluster eth0
- when the wizard window pops up, rule the test successful
4 - run ./install_cluster eth0 (again)
- when the wizard window pops up, rule the test successful
5 - e-mail the results
Again, that's a far cry from a real test, but it actually does test a
whole lot of things. We'd probably have to add a little code for the
second parts of (3) and (4) (e.g., "./install_cluster eth0
-nightly-test", where the "-nightly-test" switch will make the wizard
quit with an exit status of "0" when the wizard GUI successfully pops
up [or would have popped up]).
>From at least (2) onwards, the test would probably be best run in a
chroot-ed environment that could be reset every night (i.e., so all
the prereqs and whatnot have to be installed during (3) and then
during (4) the prereqs have to safely not install themselves again,
etc.). VMware could probably also be used, but it would probably take
a bit more effort (the test would have to be kicked off via some
init.d script, for example). This part will probably be the biggest
part of setting up the test (getting a pristine image suitable for
running this test).
Comments?
Unit tests could be constructed similarly, but that's a different
discussion...
--
{+} Jeff Squyres
{+} [EMAIL PROTECTED]
{+} http://www.lam-mpi.org/
-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
Oscar-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/oscar-devel
