On 11/18/05, John Embretsen <[EMAIL PROTECTED]> wrote:
Myrna van Lunteren wrote:
> On 11/17/05, *Daniel John Debrunner* < [EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
> The test harness can run tests within the same jvm if useprocess=false.
> This is set for the nist suite, so that all the sub-scripts are run
> in a
> single JVM, mainly for performance reasons.
>
[snip]
>
> This also implies that any tests that need special flags cannot be run
> with useprocess = false.
Are there other cases (other than the nist tests) where we should be
able to run with useprocess set to false?
To be frank, a former Cloudscape QA had at one point made it his goal to clean up and make as many tests as possible run with useprocess=false. However, that person was sent packing somewhere in the take-overs and lay-off rounds (first by Informix, then by IBM) before finishing that project. And it has not been high on anyones list since.
I think most tests *ought* to be able to run with useprocess set to false. However, this requires a certain mindset of the test writer - as it involves leaving the database behind exactly as you found it; or if absolutely required, use identifiers that are not used anywhere else in the tests. This is not always the case - we happily rely on the test harness cleaning up after us, and it's hard to think of imaginative name for a test table with an integer and character column.
As I need to use useprocess for DERBY-413, I am cleaning up the tests in derbynetclientmats (which includes jdbcapi, jdbc20 and jdk14 suites), but I'm not going beyond.
So, in short, as you found the only tests that are currently running with useprocess=false are nist.
I dug into some of the test harness code a while ago, trying to
understand a little better how the machinery works (I won't say anything
about how successful I was... ;) ).
:-) well, it works after a fashion...sort of. The whole thing could do with a clean-up, but it seems more sensible to spend time on writing JUnit tests, at this point.
I think I found other issues with
useprocess=false...
Very likely.
For example, there are a number of tests that include a call to
System.exit(), which means that when useprocess=false, the test shuts
down the JVM, and the harness is not able to process the results and
clean up after itself.
README.htm in the java/testing directory states the following:
<Quote>
useprocess
(default=true) Controls whether RunTest runs the test in a separate VM
or in a thread in harness VM. It is potentially useful for debugging
tests. Unit tests are not (yet) runnable with "useprocess=false", though.
<end Quote>
It does not say anything about which tests "useprocess=false" may or may
not work with, other than unit tests.
So I guess the question is: Is this a bug? If so, is it a documentation
bug or a bug in each of the tests listed in the attachment to this e-mail?
Not a bug perse, but if someone wants, they can open a jira enhancement & start making the rest of the tests (not including derbynetmats, derbynetclientmats, jdbcapi, jdbc20, jdk14 and multi, 'cause I'm doing those) run ok.
--
John
Executing: find ../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/ -print | xargs grep -n ' System.exit(' | grep -v '.svn'
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SpillHash.java:104: System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SpillHash.java:109: System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:312: System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:358: System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:364: System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/timestampArith.java:366: System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/CharUTF8.java:118: System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/longStringColumn.java:118: System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/testSecMec.java:81: System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:98: System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:103: System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:149: System.exit(0);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:171: System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/DerbyNetAutoStart.java:246: System.exit(1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/dataSourcePermissions_net.java:87: System.exit(-1);
../clean/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/TestDiskHashtable.java:93: System.exit( (failed == 0) ? 0 : 1);
