chromatic wrote: > I just went through a sampling of fail reports for my stuff. There was one > legitimate packaging bug, and a couple of legitimate errors due to updates to > Perl. About 35% of the other reports are these. > > I love the "Illegal seek" error message: > > http://www.nntp.perl.org/group/perl.cpan.testers/2007/09/msg602208.html
As three different reporters across five different operating systems and three versions of perl reported similar test failures, I believe the illegal seek is suspicious but incidental. > Pod::Man is broken. Think about that for a while: > > http://www.nntp.perl.org/group/perl.cpan.testers/2006/01/msg286775.html A temporary failure almost two years ago. But the six other Acme::UNIVERSAL failures appear to be legit. > No information; useless: > > http://www.nntp.perl.org/group/perl.cpan.testers/2005/07/msg221656.html > http://www.nntp.perl.org/group/perl.cpan.testers/2006/02/msg290834.html > http://www.nntp.perl.org/group/perl.cpan.testers/2005/07/msg223400.html > http://www.nntp.perl.org/group/perl.cpan.testers/2005/07/msg223401.html > http://www.nntp.perl.org/group/perl.cpan.testers/2005/07/msg223402.html > http://www.nntp.perl.org/group/perl.cpan.testers/2005/07/msg222475.html > http://www.nntp.perl.org/group/perl.cpan.testers/2005/06/msg216573.html All the "no information" reports all appear to be prior to the fixes for CPANPLUS not reporting Module::Build test results. A bug in the system that has since, I believe, been fixed about two years ago. You'll note the newest of this bunch is two years old. That leaves us with... I count one failure due to an individual CPAN tester's setup being broken with another that amounts to a warning. The rest are system-wide bugs. That CPANPLUS didn't report Module::Build test results is a highly annoying bug to be sure (in fact, the whole CPANPLUS v Module::Build war is a tragedy), but a bug that was fixed. There's nothing we can do about previous mistakes, only future ones. No use dwelling on the past, let's see how CPAN Testers is treating your current releases... * P5NCI, 11 Dec 2007... looks like a valid pile of XS compatibility issues with similar failures coming from several different testers. Valid failures. * UNIVERSAL::isa, 24 Nov 2007... looks like they caught a compatibility issue with 5.5.5 and 5.6.2 across multiple testers. Valid failures, all. And it's an alpha release, isn't it nice to have people testing your alphas? Previous stable release in Feb 2006 has one failure out of 400 tests. Looks like a valid failure, possibly due to CGI.pm not being available and the test checking for availability but not skipping the dependent test. [2] * Test::MockObject, 29 Jun 2007... 100% passing with 201 tests. Previous version from October 2006 has only one failure with 153 passes, possibly due to a bleadperl issue. Maybe annoying to you, but useful for bleadperl development. [1] * Text::WikiFormat, 29 Jun 2007... 100% passing with 71 tests. Previous version had 1 failure out of 43 tests. Again, possible bleadperl issue. * SUPER, 04 Apr 2007... one failure, a possible bleadperl bug. Depending on which way you score the bleadperl failures for your latest five distribution releases you've had zero or two false negatives out of, let's add it up... 51 + 50 + 201 + 71 + 46 == 419. So either a 0 or 0.5% false failure rate. That's a pretty damn good. [1] It can be argued that bleadperl testers should probably not email authors, and maybe they aren't I can't tell from these archives, but at least the work is useful. CPAN::Reporter could change the default configuration if it detects a development perl. [2] And before you say "how do you install perl without CGI.pm?!" it can be done with a stripped down Debian system via their perl-base package and possibly other perl distributions. -- We do what we must because we can. For the good of all of us, Except the ones who are dead. -- Jonathan Coulton, "Still Alive"
