Rupert Smith wrote:
I don't think the test clients will need to use
JUnit/CppUnit/WhateverUnit. Mostly they will just be fairly simple
scripts that react to instructions sent by the coordinator. Although,
having said that, you could use *Unit if you like.
The test clients will need a way to:
- group related tests together.
- assert expected conditions are met.
- handle unexpected failures.
- collect and report results.
- provide a natural test environment for each language
In other words the tests will need everything that the *Unit frameworks
provide. It may not seem like a lot, but would you consider writing
JUnit from scratch to test Java code? It won't be any easier to build a
distributed test framework from scratch - in fact it'll be harder and
the result will be less usable. Trust me, I've been there.
I'm just not sure
it adds anything, because these clients won't do anything when run on
their own. They need the coordinator to push them along.
Right but how do you express that on the client side? I would say you
express it as a "QpidTestRunner" test runner for *Unit tests. The
QpidTestRunner is responsible for talking to the co-ordinator. The
co-ordinator tells the test runner "Run BasicTests.getGet" the test
runner sets up the test environment executes the corresponding JUnit
test, collects the results and sends them back to the co-ordinator. The
"test client" *is* a testrunner. That means we can flexibly load as many
or as few tests as we want into the runner, run tests selectively and do
all the other good stuff that you can do with *Unit tests. It also means
the tests themselves are straight *Unit, nothing new to learn for test
writers.
It may not seem like *Unit is doing a lot in the scenario above, but
believe me, speaking as one who has participated in reinventing it
several times *it is not worth reinventing*.
The coordinator itself is a different story. I've sketched an outline
for it in Java built on top of JUnit. Trying to use JUnits existing
TestSuite mechanism, and ability to locate tests in *Test classes. A
little bit of adaption perhaps needed to dynamically name the tests,
or to run the same test case many times accross differnt combinations
of clients. A root test case called something like InviteTestCase to
provide common code to do the invite and gather reports, and provide
convenience methods for common tasks etc. Here JUnit adds a lot as a
convenient framework to base this on.
All the more reason to use it on both sides: give people a common
framwork instead of making them jump from JUnit to some ad-hoc
framework. E.g. with JUnit on both ends the client part of a test can
sit right beside the co-ordinator part and use some simple naming
conventions to make it obvious what goes with what.
I'll aim to get this skeleton code, and skeleton code for a java test
client checked in before fleshing it out. That way, other test clients
can copy the same structure.
Looking forward to it :) I'll try to put some code where my mouth is
when the skeleton is in place.
Cheers,
Alan.
