Vemund Ostgaard wrote:
This would mean that when executing a single TestCase class (or suite)
with a junit runner, the default behaviour could be to run with all
(supported) frameworks if that is what is most common. Running with just
a selected framework could be possible with a system property.
I now think this is the correct direction for JUnit tests, so that
running, say, the lang.TimeHandlingTest, would run it in embedded and
network client, not just embedded. Thus there would not be a reliance on
top-level suites to run in different fundamental configurations. I see
fundamental configurations being embedded, derby client and possible the
db2 client. Other configurations would be handed at a top-level suite,
e.g. run a set of tests with an encrypted database.
However, I'm don't see a clear way forward of how to implement this.
Currently in suite() methods the code checks the current configuration
and adds test cases based upon that information. e.g. only add this
fixture if it's embedded. Thus the assumption is that the suite() method
will be called multiple times, once per configuration.
To push the multiple configurations into the suite() I see a number of
possible options;
A) Have a standard pattern of calling the suite method calling itself
indirectly through existing decorator,
TestConfiguration.derbyClientServerDecorator.
public static Test suite() {
...
// run under other configurations
if (usingEmbedded()) {
suite.add(
TestConfiguration.derbyClientServerDecorator(
TimeHandlingTest.class);
// possible similar code for DB2 client
}
...
Seems a little confusing, because the embedded (base) configuration is
the one that adds the other configurations, the usingEmbedded() is there
to stop recursion.
Other issue is that if the test doesn't want to run in client, does it
not include the standard pattern, or just use the usingXXX() methods to
set up empty suites for the configuration it doesn't run?
B) Have a test's suite method just set up a suite as it sees fit:
public static Test suite() {
TestSuite suite = new TestSuite();
// Embedded tests
suite.add(TimeHandlingTest.class);
suite.add("embeddedTestA");
suite.add("embeddedTestB");
suite.add("embeddedTestC");
// Client tests
TestSuite client = new TestSuite();
client.add(TimeHandlingTest.class);
client.add("clientTestA");
suite.add(new NetworkServerTestSetup(client));
return suite;
}
Downsides are:
possible lack of consistency, but maybe the consistency is use of
the NetworkServerTestSetup decorator.
how to handle remote server testing?
Upside is:
clarity
matches standard JUnit pattern
non-reliance of having the configuration setup at suite setup time.
C) Something in the BaseTestClass that automatically runs the test
fixtures in multiple configurations. e.g. modifying runBase to call
super.runBase() for each configuration.
Upside is tests automatically run in all configurations.
Downside is that decision if to run a text fixture or not is based at
test runtime and not suite setup time, which means cluttering test
fixture methods with if statements to indicate if the will run or not.
Other downside is that a large number of tests may not need to run in
multiple configurations, e.g. tests that test SQL functionality may not
have much benefit running in any client mode.
-----------------------------------------------------
Are there any other ideas? Having written them out, my preference would
be B), seems very clear and straight forward. Before I wrote it out I
was thinking that none of the solutions were good.
Thanks,
Dan.
