[let's get this over to a technical list like it was supposed to be ;)] On Thu, Nov 17, 2005 at 10:43:34PM +0100, Stefano Zacchiroli wrote: > On Thu, Nov 17, 2005 at 06:43:32PM +0000, Ian Jackson wrote: > > This means execute debian/tests/fred, debian/tests/bill, etc., > > each with no arguments, expecting exit status 0 and no stderr. The
> Having been involved in various unit testing packages I found the above > expectations too much constraining. The first thing you will need after > requiring all tests not to fail is to be able to distinguish: "test that > need to suceed" vs "test that need to fail". Only the misbehaviour of > tests wrt their expected result should be reported as test failures. I > thus propose to add > Following your exit status based approach you could add to stanzas > something like: > Expected-Status: 0 > I found the above requirement the very minimum for a test interface. > What follows is optional (IMHO). FWIW, I don't see that there's a clear advantage to having the test harness *expect* non-zero exit values (or non-empty output as you also suggested). It may make it easier to write tests by putting more of the logic in the test harness, but in exchange it makes it harder to debug a test failure because the debugger has to figure out how "correct" and "incorrect" are defined for each test, instead of just getting into the meat of seeing why the test returned non-zero. Likewise, expecting successful tests to be silent means that you can rely on any output being error output that can be used for debugging a test failure. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/
signature.asc
Description: Digital signature
