Jacob Bachmeyer <jcb62...@gmail.com> writes:
> In this case, the specification is "The syscall shall return an error if
> given an invalid argument." and the test provides an invalid argument and
> expects an error return code. The test *passes* ("ok" in TAP) *if* *and*
> *only* *if* the syscall reports failure when given an invalid argument.
Right, this is the way that I've always written TAP tests. The failing
call is surrounded by some code that looks at the error return and makes
sure it fits whatever the expected behavior is for the error code, and
then emits "ok" or "not ok" based on that comparison.
It had never occurred to me that someone would want to write a test in
such a way that the failure of some operation is passed up to the test
harness and the test harness itself is told that the failure indicates a
passing test. (I *think* that's what's being described here? My
understanding is still a bit shaky.)
If that's what's being discussed, that seems like a limiting way to write
tests. Normally I would want to inspect the exact details of the failure,
make sure errors are reported correctly, and so forth, all things the
harness isn't going to know how to do. So I guess I would recommend
against writing tests that way, but there is probably some motivation that
I'm missing.
--
Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>
