25. Controlling test suite runs
Proposal:
The META spec should add flags to indicate that distribution's
tests may be shuffled or run in parallel. (SlavenRezic)
Comments:
* The prove utility provides the interesting options --shuffle and --jobs.
With --shuffle the test suite is not run in alphabetic but random order.
With --jobs multiple test scripts run in parallel. This could speed up
testing, especially on multi-processor machines.
* Unfortunately authors may write their test suites that neither one or the
other option may be used: authors may have some "initialization" and
"cleanup" script in the test suite (so shuffle cannot be used), or may
use a file or other resource in many test scripts (so parallel runs
cannot be used). So we cannot assume that --shuffle and --jobs may be
used by default.
* A sample definition could look like this (please think of better key
names):
test_run:
may_be_shuffled: yes
may_run_in_parallel: yes
Maybe even a control file to manage chunks/blocks that the module authors
already set up (Tux).
test_conf_file: t/t_conf
* I quite like this idea in principle, mostly because I've never seen a
better suggestion for how the Makefile.PL or Build.PL could make parallel
testing work. That said, it seems like it SHOULD be possible to make this
happen without having to go via metadata (which is for the configuration
to communicate to the CPAN client). Could the CPAN client really do
anything about this? --Adam K
