enhance test harness to allow control of running test by JVM id, rather than
just JVM version
---------------------------------------------------------------------------------------------
Key: DERBY-1074
URL: http://issues.apache.org/jira/browse/DERBY-1074
Project: Derby
Type: Improvement
Components: Test
Versions: 10.1.2.2
Reporter: Mike Matrigali
In looking at the growing list of issues in the regression test component, it
looks like we are seeing a number of
issues that are specific to a particular JVM on a particular JVM version. The
harness currently only allows control of
running a test either in the a suite, or not in a particular jvm version. It
seems like we have seen a need to be
able to disable a test in a particular jvm version (ie. 1.4.2) on a particular
jmv (ie. ibm version) on a particular OS (ie linux version xxx).
I believe the model we are trying to get to is that we try to reach 0 issues in
the regression test suite, ie. there are no
"expected" errors. There are always going to be bugs that don't have anyone
who is interested in fixing them, and those
should be reported in JIRA. At that point if they are not going to be fixed,
they should be removed from the particular
environment they break in, there is no point in rerunning a breaking test which
already has been diagnosed and reported.
There are currently issues being left in, because the test harness does not
have the correct granularity. Removing the test
would require it to be not run in some environments where it does pass and thus
we would lose some code coverage.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
