> * clean the source, destination directories > * pull latest CVS tip down.
Why tip? Lets simply update the current source tree to the most current of whatever branch they had checked out initially. Running it on older stable branches is just as useful. > * record environment / installed packages Virtually impossible, especially if people take to installing some items by hand (we want to test them as well). Recording the configure output on the other hand would be quite useful. > * loop - on different options ( w/ or w/o krb5, w/ or w/o ssl, etc. ) > o make clean > o configure with sets of options > o compile > + log messages > + analyze errors ( perhaps gather statitistics: > warnings, failures, notices, etc.) Any warnings, failures, etc. aside from what is in the 'known' file should be reported -- makes sense. > o (run / install) if successful > o run tests > + output results (perhaps to HTML) > + compare results with expected > + record differences if any | gather aggregate information Standard regression tests should suffice. Any non-ignored errors reported. > o uninstall / clean up Skip this. Cleanup prior to the run. If something is wrong, we may need additional information from the environment. > * end loop > > Perhaps there could be an occasion where the test would be able to put > in a corrupt WAL or a corrupt table to do regression tests for recovery > of errors. These would be interesting extensions to the standard regression suite -- but perhaps require a flag > Of course, these are just ideas and I'm not sure how practical it is to > do any of them. I just am really concerned about the uninstall/clean up > phase and how that can be done in an orderly fashion. Unless the > process can start from a clean state again, then it won't be valid. At I've not tried, but if PostgreSQL can do an 'out of tree' compile it could make it much easier. That said, poor cleanup would be a result of a broken make clean. > one point I had even given thought, vainly, to purchasing VMWare for > such an occasion. Suggestions? Skip VMWare. Find a few volunteers to cron the script (I'll volunteer). I think we should replace Bruce's pgtest script with this one -- with an argument to accept the email address to report to for FAILING cases. Success isn't very interesting if it runs regularly. -- Rod Taylor <[EMAIL PROTECTED]> PGP Key: http://www.rbt.ca/rbtpub.asc
Description: This is a digitally signed message part