On 7/2/25 13:00, Soham wrote:
[...]
Well, I forgot to clarify in the original email, but my particular target
use-case is is "unit tests" written in code, say for a particular
function or
class. In that case it gets cumbersome to have one binary per test
(atleast IMO).
My idea is to add support for a TAP output like so:
[not] ok 5 - Call sys_open with negative file descriptor # XFAIL
Where the trailing XFAIL directive would put it in the output for XFAIL,
and consider the test "passing". If it were to be an "ok" it would
count as a
failing test.
[...]
Essentially its encoding the expected-fail information in the TAP
message itself
rather than in the surrounding runner script or harness.
This is a good idea---code in the test script can determine if the test
is expected to fail and mark it accordingly. Note that expected
failures should *not* mean an expected error return: in your example of
passing invalid arguments to a system call, the test *passes* (TAP "ok")
when the system returns an error.
XFAIL is for tests that demonstrate a bug that has not yet been fixed or
that expose a bug in some underlying layer. The test really does
fail---the result obtained is *not* what it should be, but that failure
has come to be expected.
-- Jacob
