Trying to figure out why test-attach was failing pointed out an issue with the test framework.
Currently, the names of the test input/output files are hard-wired into the scripts, and use $$ to somewhat "uniquify" the names to avoid collisions. When a script runs multiple tests it looks like the same naming template is used throughout. In this specific case, check diffed the files (which failed), then the following test clobbered the files. Not having the files from the failed test at hand made it difficult to see just what the underlying failure was. I had to hardwire an 'exit 1' at the appropriate location in test-append in order to examine the data files. It would be nice if a failed test would leave its data files around for a post-mortem. There are a couple of options: 1) Modify all the test scripts to use unique filenames for each test inside the script (and ensure check() doesn't remove them on failure). 2) Modify run_test() and check() (and perhaps others?) to internally generate a unique filename prefix for each test invocation. This also requires modifying all the test scripts. 3) Modify check() to abort in the face of failure, leaving the intermediate test files behind. (1) and (2) seem like a lot of busy-work, and (2) seems needlessly bureaucratic. Given that any test failure is a build failure, potentially skipping a couple of follow-on tests in the same scripts doesn't seem all that harmful. If there are no loud objections, I'll go ahead and implement option (3). --lyndon
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Nmh-workers mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/nmh-workers
