Graham Barr wrote:
On Sep 3, 2008, at 1:01 AM, Slaven Rezic wrote:
Why does cpan testers have to ask authors to do non standard things
just to play nice with them ?
Because in this installation/testing phase there can be at least the
results PASS, FAIL, and NA (I am not sure if UNKNOWN can happen here).
Unfortunately the build script can only return a non-zero or zero exit
status. There's no protocol about the remaining values (1-255) to
differentiate the status.
Think of the exit status as a flag whether a FAIL report should be
sent or not.
I disagree. When CPAN testers was setup, a FAIL was sent if the tests
failed. If the Makefile.PL does not complete then you cannot know if the
tests fail or not, hence an UNKNOWN situation
Graham.
This behaviour would fix most of my complaints about the testing
process. Most of the bug reports I get for Net::CUPS are for:
a) CUPS isn't installed
b) CUPS is too old
I even once received a FAIL because the testers hard drive was full!
I obviously test for A and B in my Makefile generation. I would be more
then happy to get an UNKNOWN for those machines that don't meet the
qualifications for A or B. As it stands now, Net::CUPS passes 2 and
fails 9 times. I am not saying that I am the best coder in the world,
but it is a bit unfair and gives a poor representation to people using CPAN.
The only other solution to this problem that I see (besides doing away
with automated testing) is to create a framework specifically designed
to manage the build and testing process with the appropriate hooks for
automation.
Then again ... one has to ask, "What extra value does this provide?"
--
========================================================
D. Hageman <[EMAIL PROTECTED]>
Dracken Technology, Inc. http://www.dracken.com/
========================================================
