Hi Testers - I hope you're open to some suggestions from the periphery of your project. I want to help you guys make the CPAN Testers efforts more useful. One common complaint about the CPAN Testers that I've noticed is FAIL messages that module authors get that they don't want. Some people may see receiving them in their mail as a benefit, so I understand the desire to continue sending them. But, perhaps their frequency can be reduced in a few situations.
If this is covered by a plan that's in the works, I apologize. Having the reports come in by HTTP or whatever may be covering this. I keep an eye on the CPAN Tester stuff, but I'm not intimately involved in it, as I'm sure you're aware by the infrequency with which I speak up. If the smoke testers and CPAN::Reporter and the like didn't send the reports directly to module authors, but instead they went through something that processed them centrally, then there could be some more control placed on the mails that go to the authors, so the first part of the plan that I'm envisioning is to default those tools to not sending mails directly to module authors (or not allow it at all). Then, then reports only go to the cpan-testers list. >From there, there could be a decision on whether to send them on to the authors at all. Perhaps by default they are, though I don't really mind either way. * If particular authors have opted out entirely, then of course they could get no reports. * One complaint about the CPAN Testers reports is that they sometimes report failures for old versions of perl for which the module was never designed to use. (<http://markmail.org/message/47twvn4uvmfkhy2p>) I understand that module authors can specify the minimum working version in their modules, and that is arguably better than leaving it open to chance. If there has never been a passing report on any perl older than X, then don't send any more reports for older versions of perl. They got the original report about 5.6.1, they don't need one about 5.6.0. * If a module never has worked on some particular OS (even though the author has not specifically prohibited it), then don't send another failure for that OS. They have gotten that one, that's enough. * If a particular version of the module has never passed at all, don't send more than one report, even if it's a failure for a new OS or a new platform. They may don't want to know more than once that their new version had a fatal error that makes it fail for everyone. * Perhaps I really like a particular author or module. I could subscribe to failure reports for that one even though I'm not an author at all. I think that the move to not send duplicate reports from one particular smoker or CPAN::Reporter instance is great. I see this as the next step in that direction, but it does take a lot more infrastructure. It's also possible that my implementation is not the ideal one, but I think that listening to the authors when they keep asking to get less noise can make the project better. -Andy
