Hello,
I have wanted to rewrite a old canon based test to a JUnit test for a
while, but each time I get started I always run into issues that we do
not have guidelines for yet. So, I have a few questions I would like
some feedback on.
I know there are a lot of issues to consider, far more then the one I
mention here, but I hope to get a feeling of the general direction we
want to move in.
1) Should we support specific ordering of tests (within a single class)?
This feature is wanted if a number of tests operate on shared data and
each test method can be seen as a step in the test. If the order is
mixed up, the intermediate results will not be as expected and all the
tests may fail.
In general, assuming anything about the execution order of tests is
strongly disencouraged. However, since we are testing a database system,
we might want to allow for this. This feature can influence the both the
size of the test methods and the execution time for the test/suite.
2) Should we support setting up a shared test fixture for a set of tests?
This is a common issue with JUnit, and there are mechanisms to handle
it. For instance, we could let tests that require this to wrap itself in
a TestSetup instance (as described in the JUNit FAQ). Letting
DerbyJUnitTest extend TestSetup is another solution, but it might have
unwanted side-effects.
3) Many tests just want a database to work on. Other tests want to
create their own database due to specific requirements. How do we make
this easy for the developer?
4) Should it be possible to run single JUnit test/suite from the command
line by simply invoking the main method?
Personally, I think this should be possible. This require us to remove
the current use of main for running a test in the old test harness,
which I believe is doable with a few changes (in the old harness).
5) Is the ultimate goal to end up with a pure JUnit based test harness?
If this is the goal, we should design for this and accept some extra
work for plugging JUnit tests into the old test harness.
6) Where, and how, do we want to store settings for a test?
7) How is it determined which framework(s) a JUnit test should be run
in? (i.e. embedded, client, JCC)
If you have other opinions or comments related to JUnit testing, please
put them forth :)
I hope we are able to get a discussion on all or some of these issues,
and that it results in a Wiki page describing JUnit testing for Derby. I
created a page here: http://wiki.apache.org/db-derby/DerbyJUnitTesting
--
Kristian
