Hi again, In order to complete the portability series that I have been working on, there are a few improvements that I would like to propose for the test harness/suite.
1. I like to build test cases for bugs, and then layer a bug fix on top of that, so the repercussions of the bug fix can be seen. I would like to take this one step further, and record that the recorded output is incorrect. Instead of prefixing the line of output with >, undesirable output could use ? instead. test cases for bugs can then be committed as soon as they are reproducible, found using grep, and worked on at a later date. Does anyone object to having the bug test case being committed prior to the bug fix? 2. As part of my original attempt to port quilt to msys, I found a number of binaries were producing slightly different messages to those in the test cases. The worst offender was patch-2.5. While it would seem a little anal to try to attempt to support such a historical release, it works fine and msys is a pretty finicky platform to maintain packages for, so I dont expect a new version of patch being able to compile on it for quite a while. Also, I am sure that sometime in the future, a binary that quilt uses will decide to emit a slightly different error message, and it will break the test cases. In order to address this, I propose the following changes to the test harness: 2.1 regex support for output lines that commence with >~ 2.2 failed commands to use the separators !~, =~, != and == instead of | and ? 3. local-quilt-x.diff introduces a 'make check'. This becomes very important as more of the ./configure information becomes necessary to accurately test the quilt built in the local workpit. Is it ok to remove test/Makefile? The only alternative that I can see is to replace it with test/Makefile.in. 4. regularly running all of the tests on msys is really slow. I would like to record which tests have run to completion, and pick up the testing on the last test case to fail. I wrote a rough patch to do this last year; any suggestions on how this should be done in order to be committed? http://zeroj.hda0.net/quilt/patches/test-make-depend.diff 5. Also, when one command in a test case fails, the rest are sure to follow. I would like to change the test harness so that it only continues past the first error if an option (--ignore-error) is enabled. I think this behaviour should be disabled by default. -- John _______________________________________________ Quilt-dev mailing list [email protected] http://lists.nongnu.org/mailman/listinfo/quilt-dev
