Daniel John Debrunner wrote:
Myrna van Lunteren wrote:
I want to point out that to run platform tests it's not always
possible to use ant - not all OSs or machines I try to run on during a
platform test have ant available.
So if another solution can be found than relying on junit-all I'd
appreciate it.
It's not really relying on ant, the junit-all target just provides a
mechanism to run a set of tests, just as suites.All provides a
mechanism to run almost the same set of tests.
I think a problem here is that the naming implies that both "junit-all"
and "suites.All" will run all the tests, at least that has been my own
misconception. Apparently "ant junit-all" runs some tests that are not
run by "junit suites.All", and I also think "junit suites.All" runs some
tests that are not run by "ant junit-all". Because of this there is no
easy way to run all the tests, and it is not trivial to figure out which
tests gets run when I use this target but not when I use that. Ideally
it should be easy for someone that just joined the community to figure
out how to run all the tests, to qualify Derby on a different OS or jvm
for instance.
There's no requirement for anyone to use ant junit-all or suites.All.
In addition there's nothing stopping anyone from contributing another
mechanism to run tests, e.g. a script file, a different grouping of
tests etc.
The advantage of using ant or a Junit suite is that they are standard
mechanisms, and thus the tests can be run by other tools easily, such
as Elipse, junit test runners or cruise control.
I agree, this is a good thing.
Still I think it would be valuable to make clear if two mechanisms for
running the tests are intended to be equivalent, and if so try to
synchronize their behavior as much as possible. For instance, I believe
the "ant junit-all" mechanism requires significantly less memory for the
jvm than "junit suites.All" does, as it executes the different subsuites
in suites.All in different jvms. I could add a new test and run it
successfully with "ant junit-all", but it might break for anyone using
"junit suites.All" because of different memory constraints. If I
contribute a patch I would not want the mechanism I use to run the tests
possibly hide problems that will hit others that run the same tests but
with a different mechanism, to the extent possible.
So, I think it would be good to work towards tests being executed in
environments that are as similar as possible, and in the same order,
etc. when using different mechanisms that are intended to be equivalent.
Vemund
