I've had this patch series half-done in my tree for a long time,
and have now taken the time to fix the missing bits.
The old testsuite is, well, old, and unloved. But it also covers
crucial aspects, so as long as there's no rewrite, we might as well
enable some parallelism and have a decent log creation mechanism (both
through Automake 1.11's parallel-tests feature).
Now, there are a few stumbling blocks: some tests depend upon prior
tests, some are run more than once, lots of them use the same trial
directories and files. Serialization can be achieved by adding
dependencies between tests and renaming/wrapping; parallel-tests runs
each test name at most once, given that it uses make rules to convert
tests to log files.
It would be possible to de-serialize for example the demo-static,
demo-conf, and demo-shared test groups by running them in different
build directories and installing them in different prefixes. In the
patch series, I chose (for now) not to go this route yet. We can still
go this way later. For now, this means that effectively, all the
demo-*.test tests are serialized.
Running a subset of the tests gets a little harder this way: since the
dependencies specify a total ordering, running, say, demo-shared.test
lets make think it requires running a bunch of others first. The GNU
make feature to specify order-only dependencies can alleviate that a
bit, and the patch series tries to exploit this when possible; if you're
not using 'make' for building, you would need to pass e.g., MAKE=gmake,
to configure to have this enabled.
Finally, the series improves the 'make check{non,}-interactive' rules
to also apply to the old testsuite as well, to hopefully benefit MSYS
and Cygwin Libtool developers.
I have tested the patch series (up to trivial changes) on several
systems, including MinGW, Cygwin, AIX, GNU/Linux, FreeBSD.
parallel-tests requires a new Automake for building the Libtool package
itself. Users of Libtool macros should still be able to cope with
Autoconf 2.59 and Automake 1.9.6. I tested this on GNU/Linux.
If requiring Automake 1.11.1 is too strict, then it would be possible,
alternatively, to only enable parallel-tests iff Automake is new enough.
This would make both code and documentation a bit more complicated, so
I'm hoping that it won't be necessary.
Feedback welcome. OK to merge?
Thanks,
Ralf
