Hi Eric,
thanks for bringing this issue to a wider audience. M::SD could *really*
use some reasonable tests.
Eric Wilhelm schrieb:
[...]
What's the best way to test this sort of thing?
I could certainly run some simple checks on core modules, but I envision
myself quickly getting into difference between platforms and perl
versions.
It seems like the best way to get traction quickly is to throw some
real-world data at it and do some reasonable checks on the results.
e.g. we could easily generate a known set of required modules for each
module in the test set. However, the "who used what" data is currently
incorrect in a few cases, so a full expect test would break after a
bugfix or two.
So, where to get a good set of perl program data to test against?
Bundle a static set of modules with it? Create an optional "data pack"
to drive the testing?
I don't really want to write-up a bunch of Foo.pm and Bar.pm modules,
particularly since what we're dealing with is a rather large and
feature-packed system that's gone through quite a bit of fix-this-
here-and-that-there.
I would guess it's prudent to select a few core or CPAN modules that
meet the following list of criteria and put them into t/testlib. Then
run M::SD against various combinations of them. Do it once by hand and
dump the resulting dependency structure. Have a quick look at their
source to grasp the obvious dependencies and check that the structure is
correct-ish. Expect that (possibly hand-corrected) structure be returned
by future ScanDeps runs. One more thing: It is very important that
"t/testlib" is added to the front of @INC so M::SD picks up the right
versions of the test modules.
- pure-Perl
- backwards-compatible to at least 5.004
- no non-core dependencies that aren't also in testlib
- no dependencies on core modules which changed their dependencies
between 5.005 and 5.10
- for core modules: no changes in dependencies between 5.004 and 5.10
I know it is probably hard work to match these criteria, but it's
necessary because otherwise, the tests break on some systems depending
either on their perl version or whether they have some module installed
or not. Now that I think of it, if the core modules that are loaded are
pure-perl and work across a wide range of perl's, it would be feasible
to put them in t/testlib, too, if their dependencies changed between
5.004 and 5.10.
If available, a couple of tests for some of the special-cased CPAN
modules could be run (like there is a test for Module::Pluggable). I'm
particularly thinking of Tk here.
Does that make sense?
Perhaps it's easier/less work to write Foo.pm/Bar.pm by hand after all.
Steffen
