On 7 Mar 2007, at 21:15, Eric Hacker wrote:
Monitoring would check that the http server is up, and check that the
database server is up, but this is functional testing. Does the
application work, end to end.
OK, /now/ I understand.
That's what Test::More does. Otherwise, everything would just be Ok.
Sure - but that's human readable information. As far as the harness
is concerned it's succeed or fail plus a bunch of text it passes
through verbatim.
In summary I think you're saying "don't just tell me it failed, tell
me /why/ it failed dammit" and we're saying pretty much the opposite:
"just tell me if it worked or not and leave it to me to find out
why".
is_deeply and cmp_ok start to say why.
Now why is a matter of degree, and if a test could tell that the
developer meant && instead of || or that the new HTTP firewall was
dropping all POSTS with "hacker" in the data, then we'd be out of
work. I thought the spirit was to present as much info as practical.
Sure, absolutely.
Correct me if I'm wrong (again) but you were advocating returning an
HTTP-like status code per individual test, yes? Given that there
innumerable ways for a particular test to fail I can't quite see how
you can distil that down to a status code.
I really don't care too much, because I am already making what's there
work for me now. But if you are asking what to improve... :)
Thanks for taking the time. I appreciate that even if I can't
actually work out what you're advocating :)
I took a look at http://www.testobsessed.com the other day. Check out
the functional test tools next generation dream. That makes status
codes look like Lego Duplos.
I'm clearly having a stupid day. Is this what I'm supposed to be
looking at:
What Problem Would Next-Generation Functional Testing Tools Solve?
http://lyxus.net/gdc
I can't see anything there that's much of a reach for us.
--
Andy Armstrong, hexten.net
