Hello Perl QA Wizards,
I apologize in advance if this is documented somewhere, because I must
have missed it.
Is it OK to have the return value from a test be something more than
just true when a test passes? Thus the test might be used as a right
value in the test script.
Background:
I am working on a test suite to support distributed functional
testing. I have a test 'like_body(<request>, <expected>, <name>)'. It
makes a request of another system and compares the result returned
from the other system with expected.
There may be information in the result that does not affect the
outcome of the test, but would be useful to have for a future test.
A specific example:
The function being tested is a web security application.
Alice is running a web server and the test script. Bob is set up to
attack Alice from random IP addresses. In the test script Alice asks
Bob to visit a url:
$bob->like_body('tget url http://alice/nasty_url resp=404 nat_ip',
qr(^ok), "Nasty_url not found");
If this test passes, we know that Bob got a HTTP 404, but not from
where. It could have been Alice or the security application. We need
another test to look in Alice's logs to see if the query came in, and
she needs to know the source IP address to make sure that we're
looking for the right query. The nat_ip parameter in the request
indicates to Bob to return the source IP address used with the
response.
Now, I know I can just make the request outside of a test and then
process the results within a test, but I lose some diagnostic elegance
that way.
I'm somewhat comfortable with the object syntax required to make
requests of different systems. It has either got to be
$bob->like_body(...) or like_body($bob,...) and I'm leaning toward the
former as being clearer.
However to make the test a right value does not seem in line with the
general expectations of how test scripts are written. $resp =
$bob->like_body(...) does not look like a test to me anymore. Perhaps
I am just being too particular or just haven't seen enough test
scripts, though. So I ask, what do you think?
Thanks for all the hard work in making Perl, TAP, etc. the excellent
foundation from which to build a tool to fulfill my testing need.
Regards,
--
Eric Hacker, CISSP
aptronym (AP-troh-NIM) noun
A name that is especially suited to the profession of its owner
I _can_ leave well enough alone, but my criteria for well enough is
pretty darn high.
