On Wed, Sep 03, 2008 at 10:25:46AM -0700, chromatic wrote: > On Wednesday 03 September 2008 00:35:53 Barbie wrote: > > > cpan-testers are reporting what a user would experience using CPANPLUS, > > but not having Module::Build, and having problems with installing > > Module::Build. While you may not be interested in being made aware of > > those potential failures, others are. > > Let me change directions and ask a philosophical question. I suspect the > answer will be revealing. (No, this is not a logic trap. I think this is at > the heart of disagreements over the utility of certain CPAN Testers reports.) > > Is the purpose of CPAN Testers to report whether a distribution will install > on a given machine or whether its tests pass on a given platform?
In strict terms, No and Yes, but that is only a small part of the picture. The more detail answer is as follows: Does any part of CPAN Testers test an install? - No Do CPAN Testers verify a test suite passes on a given platform/perl? - Yes Does CPAN Testers verify that the distribution can be built on a given platform/perl? - Yes Does CPAN Testers verify that the build/make stages actually work? - Yes However, again that still isn't the whole picture. Testing on a near clean system (and by clean I mean only the core, although the CPAN Testers tools do require a few extras), has helped identify when authors have missed prerequisites for their lists in either the build file or META.yml. Some who are testing as they go along and install stuff don't see these types of problems. Adam Kennedy did try and identify several thousand environment scenarios that he could generate PITA images for, which would be interesting, but for many authors, such as yourself, would likely hold no value at all. For you, you are also interested only in getting the user to use the last of everything. If that's not true, then that is the impression you're giving. That doesn't work in all cases. At MessageLabs we are still having to use 5.8.0 because so much depends on that version. Many of us have been trying to get a more recent build accepted, but it's extremely hard due to the vast amount of servers around the world we have. Upgrading is a fragile process and for a business like ours it can bring down the company if it goes badly wrong. Other users have similar stories. Part of the purpose for CPAN Testers is to try and indicate whether a particular distribution can pass with a given platform/perl, but more specifically what versions pass with that given platform/perl. In some cases the latest version fails, but an earlier version passes. CPAN Testers doesn't guarantee that a particular distribution/version works, but that it passes the build, make and test suite. It may well not be enough in some cases, but it's currently generally a good indicator. At the moment we don't cover enough platforms, and that is where I would rather encourage people to get involved. There are plenty of things we are currently working on that will reduce the email traffic, as well as helping to provide reports that are appropriate. There are several of us who have been trying to promote the changes and improvements, and there has certainly been opportunities for people to feedback ideas. Instead of announcing some of those changes and improvements, I feel completely deflated and very unimpressed at effectively beaten up for trying to make a difference. I take it personally because I put in several years to try and encourage people to get involve, contribute ideas for improving CPAN Testers, and generally make it something of value to everyone. I just hope the likes of David, Ricardo, Chris and David don't get as deflated as I do right now. Barbie. -- Birmingham Perl Mongers <http://birmingham.pm.org> Memoirs Of A Roadie <http://barbie.missbarbell.co.uk>
