# from Paul Johnson
>How is "didn't do what I expected" *ever* any sort of success?
It *did* do what the *test* expected ("ok(1)"). Yes, the TODO expected
it to fail, but if it is passing, you have "more than success".
It might need attention, but "failure" is a subset of "needs attention"
or "unexpected", not vice-versa.
>Just playing devil's advocate here really, but experience has taught me
>to be rather conservative when it comes to tests and that I'm not
>clever enough to predict all the things that can go wrong, so the fewer
>I let through the better.
$who lets $what through $where? Tests on the CPAN have many contexts,
so defaults need to work in all of them and any options in the harness
should do the right thing in most of them. Feel free to apply your own
local policy as far as whether to ship with passing todos or not, but
calling it "FAILED" in the harness is the wrong way to go-about that.
If it were an option to report "extra success" as "failure", I'll bet
that we'll end up with one or more cpan testers having it on
(inadvertently or not.) Then the red 'failed' lights go on for a given
platform (because it happens to work *better*) and we're left
saying "what the hell were we thinking?"
We don't report skips as failure, even though some skips actually
mean "missing features" or "unable to test in this context" -- which is
a case of "less success" than a todo passed.
Finding humor in the irony is left as an exercise for the reader.
--Eric
--
We who cut mere stones must always be envisioning cathedrals.
--Quarry worker's creed
---------------------------------------------------
http://scratchcomputing.com
---------------------------------------------------
