I just committed HarnessJavaTest which is an experiment in running an
old harness style '.java' test using the JUnit infrastructure. The old
harness test (e.g. jdbcapi/connectionJdbc20.java) is unchanged and pass
is indicated by comparing with its master file. This is very similar to
the running of .sql scripts through JUnit using ScriptTestCase /
LangScripts / NistScripts. As in those tests only a single master file
is supported and no 'seding' takes place.
I also added a concrete implementation in JDBCHarnessJavaTest which runs
'.java' tests from the jdbcapi package. It ran 17 jdbcapi '.java'
without changes in embedded. (And a subset of those with client/server
though that is not enabled in the committed version, I just need more
investigation). It is not added in as part of suites.All and the tests
listed in it continue to be run with the old harness as part of derbyall.
I'm looking at this approach as a quicker way to moving to a single
(JUnit based) test infrastructure. While many tests have been converted,
conversion of an individual test is slow, and thus the overall
conversion rate is slow. The current number of tests to covert is still
high, see:
http://wiki.apache.org/db-derby/KillDerbyTestHarness#head-a6e2c89e4bc09edf99f4a7bb6e12af333b6d01f2
While canon based tests are not ideal, I think they are acceptable as an
interim step, especially if we manage to switch to a single
infrastructure. A good goal would be to have the 10.3 release (mid
year?) with a single infrastructure so that bug fixing and text cases
are easy to merge between trunk and 10.3. Also running the tests in this
way does not stop anyone from converting an individual test, it makes it
no harder. In fact it *might* make it easier, as a it might be possible
to incrementally convert tests and have them continue to run with a
master file.
One benefit I see from this approach is that it may allow the community
to focus on converting tests that cannot be run using this or other
adapters, for example ones with multiple canon files, per jvm version
for example. Those tend to be the tests that occupy the most time as the
need to update multiple canons is costly.
It will also centralized which environments a test runs in, one of the
issues I'm seeing with the jdbcapi tests and client server is trying to
figure out if a test is meant to run in c/s or not. Not clear from the
old harness setup.
I'm going to see if I can get a similar high pass rate with .java tests
in another folder, probably store. If so I will plan on changing the
tests that can be run with an adapter to run as part of suites.All and
not derbyall. I would also create a wiki page that listed all the old
tests and if they are to be run with the harness or JUnit, as it could
get hard figuring out what state an old harness test is in.
Feedback, thoughts?
Dan.
PS. HarnessJavaTest was surprisingly easy, the only setup required by ij
to run one of the old tests was to set the ij.database system property.
For J2ME different properties will need to be set, but that's only a few
lines of code.
