On 19 Dec 2007, at 02:59, Chris Dolan wrote:
Presumably the false negative rate achieved by the best modules is
a measure of how noisy the smoking system is. Given that the
cleanest modules regularly get a <1% FAIL rate over many tens of
reports it's not a huge reach to suggest that any module should be
able to get close to that. So I'd expect an author who was seeing a
lot of failures to look around on CPAN a little and observe that
other people's tests are doing better than theirs and then maybe
wonder whether it's their code that is at fault.
Does anyone know how the false negative rates compare for cpan-
tester smokers vs. CPAN::Reporter users? I've found the former to
be enormously valuable for cross-platform testing (especially David
Cantrell and Slaven Rezic), but I have seen very little feedback via
the latter at all.
Overall I'm seeing a noise floor of <1% but I'm not sure (and would be
interested to know) what the split between CPAN::Reporters and smokers
is overall.
I've just grabbed a pseudo random sample of my test results:
CPAN::Reporter: 15
Smokers: 13
Possibly not representative but it certainly implies that authors will
be seeing a mix of results from both sources.
--
Andy Armstrong, Hexten
