> Have you tried webchat? You can find webchatpp on CPAN.
Just had a look. It appears to be a rehash of chat (expect) for the
web. Great stuff, which is really needed and demonstrates the power
of Perl for test scripting.
But...
This is a bit hard to explain. There are two types of XP testing:
unit and acceptance. Unit testing is pretty clear in Perl circles
(ok, I have a thing or two to say about it, but not now :-).
Acceptance testing (aka functional testing) is traditionally handled
by a third party testing organization. The test group writes scripts.
If they are testing GUIs, they click in scripts via a session
recorder. They don't "program" anything. There's almost no reuse,
and very little abstraction.
XP flips testing on its head. It says that the programmers are
responsible for testing, not some 3rd party org. The problem I have
found is that instead of programming the test suite, XPers script it,
using the same technology that a testing organization would use. With
the advent of the web, this is a real shame.
HTTP and HTML are middleware. You have full programmatic control to
test your application. You can't control the web browser, so you
still need to do some ad hoc "how does it look" testing, but this
isn't the hard part.
The acceptance test suite is testing the system from the user's point
of view. In XP, the user is the customer, and the customer writes
tests. In my opinion, this means the customer writes tests in a pair
with a programmer. The programmer's job is to create a language which
the user understands.
Here's an example from our test suite:
Accounting->setup_investment('AAPL');
The user knows what an investment is. She also knows that AAPL is a
stock ticker. This statement sets up the environment (using LWP to
the app) to execute tests such as entering dividends, buys, sells,
etc.
The test infrastructure must support the ability to create new
language elements with the ability to build elements using the other
elements. This requires modularization, and today this means classes
and instances. There's also a need for state management, just like
the request object in your web application.
Part of the packaging process we're going through is making it even
easier to create domain specific languages. You actually want to
create lots of dialects, e.g. in our case this means investments, cash
accounts, member accounts, and message boards. These dialects use
building blocks such as logging in, creating a club, and so on. At
the bottom you use LWP or webchat. However, the user doesn't care if
the interface is HTTP or Windows. You're job as a test suite
programmer is meeting her domain knowledge, and abstracting away
details like webchat's CLICK and EXPECT OK.
In the end, your test suite is a domain knowledge repository. It
contains hundreds of concise scenarios comprised of statements, or
facts, in knowledge base parlance. The execution of the test suite
asserts all the facts are true about your application. The more
concise the test language. The more easily the user-tester can verify
that she has encoded her expertise correctly.
Rob
