----- Original Message ----
From: Michael G Schwern
> Why do "ok # TODO" tests pass? Users will report
> "the tests failed and the CPAN shell won't install it
> and now I can't install and
> OMGWTF your shit is broke" when in fact it works
> even better than the author thought!
<snip>
> In toolchain land, RFC 1925 Section 2.1 dominates.
For those confused:
2. The Fundamental Truths
(1) It Has To Work.
The CPAN shell isn't tcp/ip, it's udp.
CPAN and CPANPLUS fail to install things all the time even though there are no
test failures in the author's code. Remember the 'Sub::Uplevel' crap folks
were dealing with? Plenty of modules fail to install because of false
negatives. So plenty of users fail to install good modules. Other users get
used to 'force' installing them. The savvy ones dig in to find out why the
failure occurred and they decide whether or not to install. If we had a
reliable toolchain, this wouldn't be an issue, but we don't, so it is.
Now, instead of a false negative, I'm recommending a false positive. However,
as you've pointed out, this is merely a parser. I present all of the
information necessary to allow the end user to write their own custom harness,
so if they want 'ok # TODO' to pass, so be it.
Cheers,
Ovid
