Ovid wrote:
> --- Michael G Schwern <[EMAIL PROTECTED]> wrote:
>
>> The whole idea of halting on first failure was introduced to me by
>> some XUnit
>> folks ... As any field scientist knows, there's no such thing as
>> uncontaminated data.
>
> As any tester knows, a "one size fits all suit" often doesn't fit. Let
> people decide for themselves when a particular method of testing is
> appropriate. I hate "you must halt testing on a failure" as much as I
> hate "you must not halt testing on failure". It's not XOR.
When it comes to failure, I like to err on the side of more information.
> There's a certain irony that beginning testers are often told to fix
> the *first* error *first* and subsequent errors go away. I'm not
> saying this is a silver bullet to solve testing, but sometimes it's
> very useful.
That's the general idea for dealing with syntax errors, too.
The trick is, you don't know ahead of time whether the information from the
follow on failures will prove to be useful. Can't tell until you see it. So
don't freak out over all the subsequent failures, fix the first thing and
re-run is a decent plan, but you can't just ignore them either.
> I am feeling a bit stupid because I can't figure out your conclusion.
> Humor me. At times it sounds like you're telling people not to do this
> and at times it sounds like you're telling people it's hard to do with
> Test::Builder :)
Yes, I'm saying both. I don't like it AND it's appears impossible to do right
with TB. Though I do still ponder how to make it work anyway.
PS Couldn't you have the TAP harness kill the test process on first failure?
--
24. Must not tell any officer that I am smarter than they are, especially
if it’s true.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/?page_id=3
