On Jan 29, 2013, at 03:39 , Marvin Humphrey <[email protected]> wrote:
> Dreaming big dreams... I'd love to have a test harness that supported test
> classes like xUnit, but ran tests that don't throw exceptions, a la Perl tests
> instead of xUnit assertions. Most of the time, a failing test does not
> invalidate subsequent tests -- and having additional information from tests
> which either succeed or fail can be a big help when trying to debug a CPAN
> Testers report where you don't have access to the machine on which the tests
> ran.
Another nice feature of TAP is that you have to split your tests into separate
programs. So if one of the test programs crashes, the others are not affected.
I thought about using TAP for testing the C bindings, but we'd also have to
provide a TAP consumer like Perl's 'prove' utility to interpret the tests. For
now, I decided to go with pluggable formatters for the test output and provide
a TAP formatter and a custom Clownfish formatter. The latter can be used to
print a test summary from the test program directly.
>> +$test->run_batch('Clownfish::CFC::Util');
>> +
>
> One thing I've wanted to do for a while is have our C tests return true on
> success, false on failure, and then use that return value to determine the
> exit value for the test script. Right now our C-based test scripts still exit
> success even when some tests fail.
I just pushed a commit with some major reworks which also address this issue.
Another thing I'd like to have is some kind of central registry for tests, so
we don't have to maintain code like the following in every host language
binding:
http://s.apache.org/n3
For the CFC tests, I simply put all the needed information in a global array.
Nick
