Two tickets on CPAN::Reporter have prompted the question of the role of CPAN Testers in capturing build failures versus test failures -- i.e. failures in Makefile.PL/Build.PL or in make/Build.
* http://rt.cpan.org/Public/Bug/Display.html?id=21691 * http://rt.cpan.org/Public/Bug/Display.html?id=21566 Right now, CPAN::Reporter only wraps the "test" action in CPAN.pm and CPAN.pm doesn't even run tests if the make process wasn't successful. It wouldn't be terribly hard to add support to capture the earlier stages, but I'm not sure if CPAN Testers is the right place to capture it and, if so, how it should be coded. If failures happen before tests are run, should that be counted as an "NA" -- meaning that tests could not be run on that platform? Generally, I've seen that to mean that prerequisites could not be satisfied. Or should there be a new failure category "BUILDFAIL" (or "WONTBUILD") that distinguishes these cases? Or something else entirely Before I hack something into CPAN::Reporter and CPAN.pm, I'd like to get some community consensus, particularly from those involved in CPAN Testers. Regards, David Golden P.S. As a side note, Chorny (from win32.perl.org and #win32) mentioned that he's looking into HTTP transport for Test::Reporter -- that will be a nice upgrade for getting reports from behind ISP firewalls.
