Hi Vincent!
I think your classification is well thought, but you engrain the test
method inside each category. As a matter of fact, we use mock objects
for functional testing, what you call '3: end to end tests'; and it
should be possible to use different test methods for every category.
What follows is a (long!) description of our current testing
methodology, or at least our part in it; acceptance tests are the
responsibility of the customer, and they usually do it manually. We
don't do EJB's, since we have no EJB container.
On a well-architectured system, the servlet should contain the minimum
logic, and delegate all processing to other classes. Suppose that,
according to the functional specification, our system must GET a
parameter 'account', look it up in a DB, and then display the money in
that account. Object-challenged folks would write the following:
class NonObjectServlet extends HttpServlet
{
public void doGet(HttpServletRequest request, HttpServletResponse
response)
{
String account = request.getParameter("account");
... // connect to DB stuff
// get the money from the DB
int money = resultSet.getInt("money");
// now we write the page
PrintWriter writer = response.getWriter();
writer.println("<HTML>");
... // various HTML stuff
writer.println("Your money: " + money);
... // end page
}
}
where I have ommitted non-relevant stuff. But, an object-conscious
developer will write an architecture of classes to do the processing:
class ObjectServlet extends HttpServlet
{
public void doGet(HttpServletRequest request, HttpServletResponse
response)
{
// get the account from the request
Account account = parametrizer.extractAccount(request);
// get its info from the DB
AccountInfo info = accessorDB.getInfo(account);
// construct the page with the Pager class
String page = Pager.makeAccountPage(account, info);
// and now, print it
PrintWriter writer = response.getWriter();
writer.println(page);
}
}
where the responsibilities have been shifted to classes Parametrizer,
AccessorDB, and Pager (instances of which are kept in the servlet).
The point is, now AccessorDB and Pager can be unit-tested
in a simple way, just putting in fake accounts and account info. These
would not be mock objects, but test data that should be inserted in the
DB.
But to unit-test the Parametrizer class, we'll need a TestRequest (such
as the one I sent on monday). This makes up your 1st category.
Then, when all the pieces are put together, we'd do a functional test,
or end-to-end test as you put it (3rd category):
class ObjectServlet extends HttpServlet
{
public void doGet(HttpServletRequest request, HttpServletResponse
response)
{
...
}
static public void main (String[] args) throws Exception
{
// initialize the servlet
TestConfig config = new TestConfig();
ObjectServlet servlet = new ObjectServlet();
servlet.init(config);
// mock request
TestRequest request = new TestRequest();
// we place the parameter
request.setParameter("account", TEST_ACCOUNT);
// mock response -- will be written on file
TestResponse response = new Response("output.html");
// make the call
servlet.doGet(request, response);
// now, for a nice end-up, we check the file
HTMLChecker.checkFile("output.html");
}
}
This is a *functional* test, since it checks all of the system -- and it
also uses mock objects. To be honest, we should also perform test cases
for empty parameter, no parameter... and check that the system works
(i.e. doesn't throw naked exceptions). And, we usually do so.
You might think that it still requires developer interaction, since the
output file .html should be studied by a human to check that it really
contains the account info. In fact, it does; but only the first time
it's generated. Later, you just check that no exceptions arise -- and
this way you can catch 90% of errors introduced.
The next point is to test the system-wide behaviour, including sequences
of pages. This can also be done manually, or automatically, and would be
part of the systems testing (your 4th category); and it cannot be done
with mock objects.
I cannot say that I fully understand your scheme, though, since I don't
really see the need for a 2nd category, between unit- and functional
testing. But my point is (finally! :) that you can use different
techniques at different levels, and they would still fit that category
nicely.
Un saludo,
Alex.
Vincent Massol wrote:
>
> In order to be able to better discuss Mock Objects vs In-Container, etc ...
> I'd like to try to categorise ways of testing.
>
> The first definition that we need to agree on is that of Mock Objects which
> must not be mixed with Stubbing.
>
> Mock Objects vs Stubbing :
> - Mock Objects (as defined in
> http://www.sidewize.com/company/mockobjects.pdf) which are used to finely
> unit test a given method by isolating it from it's surrounding (this is done
> by mocking all manipulated domain objects).
> - Stubbing, which is what you describe. It can be used in several different
> places, to mock java beans in order to easily unit test JSPs for regression.
> It can also be used for example to mock an EJB so that you can unit test
> your presentation objects, ...
>
> Then, I would categorise testing into 4 different areas :
> 1- fine-grained unit testing, which is what Mock Objects do. This only unit
> tests method by method without testing object interaction,
> 2- coarse-grained unit testing, which is what Cactus does for example. The
> unit tests encompass domain objects and test interactions. Stubbing should
> also be categorised into this category.
> 3- end to end tests, some kind of functional tests but whose goal is not to
> test a complete functional use case which might involves a user interaction
> over several screens for example. Rather it tests from end to end on a
> technical feature, like testing that a JSP effectively returns the bank
> account figures.
> 4- acceptance tests or full functional tests, which test the real user
> testcase, ie. for example, the user fills a form, then if it does not enter
> the postcode, such page with such error appears, ...
>
> Do you agree with this classification ?
>
> Thanks
> Vincent.
