> You might notice a small increase in list traffic over the next few
> days. CPAN::Test is undergoing a shakedown cruise. At the
> moment, it
> will only report PASS or NA results. The mechanisms for reporting
> failure still needs some work before I'm comfortable putting it on
> automatic.
>
> It's all downhill from here. :)
Great, I was just starting into the same direction... :-)
What I'd like to know: how have you solved the problem of interactive
configuration that some modules need? Or do you leave the 'perl
Makefile.PL' step interactive and automate only the 'make; make test'?
What about dependencies?
Regarding failure, I don't think this should be completely automatic.
The testsuite should collect all non-(PASS|UNKNOWN) and present them to
the tester in a compact form for examination. The tester then decides
to either just send out the result or to investigate further based upon
output from 'make' and 'make test'. That way you can start testing in
the background and only have to put your attention to it when the test
run is finished, and then only the (FAIL|NA)s have to be examined,
which should be only a few.
I'm fantasizing about creating a real testserver, which would
automatically scan the mailing list, do the tests and report back the
results (after the aforementioned manual scrutinizations for FAIL|NA).
You'd set up a testuser account with its own private perl version (so
you can just install each and every new module), subscribe it to the
testers mailing list and start the test server, which scans the inbox
(with IMAP and maybe POP support), does the tests, sends out
PASS|UNKNOWN by itself and forwards FAIL|NA to a human tester. If the
testserver is written well, it should even be able to cope with a new
perl release and automatically upgrade itself!
Which brings me to another point: right now, only the package name is
sent out to the mailing list. This means that a tester has to wait a
few hours or days until the nearest CPAN repository is in sync and the
package can be downloaded from there. How about attaching the package
onto the announcement mail, at least if the package size is smaller
than a certain amount? Certainly wouldn't be good to send megabytes
around, but most packages are only a few kB. Probably should be a
different mailing list, so people can decide if they want the source or
not. This would help the automated testers a lot and make near-realtime
response possible.
And this would also allow to submit not-yet-released modules and have
them automatically tested beforehand.
Does this sound sensible? Or am I still asleep and dreaming?
Roland
--
[EMAIL PROTECTED]
> Long time ago, there were only one gender cells. Then one day, a
cell was infected with a disease, a DNA disorder, and reproduced
offsprings of male and female (two different kinds of gender).
Life is a sexually transmitted disease... ;-)
