Re: LargeCodegen test fails with derbyclient while running largeData test suite

[Ramandeep Kaur]

Hi,

 

I ran largeCodeGen as following:

 java -Dverbose=true -Djvmflags="-mx512M -ms512M" -Dframework=DerbyNetClient
org.apache.derbyTesting.functionTests.harness.RunTest lang/largeCodeGen.java

With the above, I did NOT get usual error which was

*** Start: largeCodeGen jdk1.4.2 largeDataTests:largeDataTests 2006-04-29 08:30:04 ***
27a28
> JVMST109: Insufficient space in Javaheap to satisfy allocation request
Test Failed.
*** End:   largeCodeGen jdk1.4.2 largeDataTests:largeDataTests 2006-04-29 08:32:01 ***

 

Instead, I got another error which is

< PASS: IN clause with 97000 parameters
20 del
< PASS: PREPARE: IN clause with 98000 parameters
21 del
< PASS: IN clause with 98000 parameters
22 del
< FAILED QUERY: IN clause with 99000 parameters.
22a19
> FAILED QUERY: IN clause with 97000 parameters.
Test Failed.

 

Here are the contents of test.txt

*** Start: largeCodeGen jdk1.4.2 DerbyNetClient 2006-05-02 17:06:24 ***
Initialize for framework: DerbyNetClient
java -mx512M -ms512M -Dderby.system.home=/local0/NightlyBuildResults.2006-05-02/
ibm142_largeDataTests/DerbyNetClient/largeCodeGen - Djava.security.manager -Djava
.security.policy=/local0/NightlyBuildResults.2006-05-02/ibm142_largeDataTests/de
rby_tests.policy -DderbyTesting.codejar=file:/local1/cloudtst/dev/src/jars/insan
e/ -DderbyTesting.codedir=/local1/cloudtst/dev/src/jars/insane -DderbyTesting.se
rverhost=localhost -DderbyTesting.clienthost=localhost -DderbyTesting.codeclasse
s=file://unused/ org.apache.derby.drda.NetworkServerControl start
Attempt to shutdown framework: DerbyNetClient
19 del
< PASS: IN clause with 97000 parameters
20 del
< PASS: PREPARE: IN clause with 98000 parameters
21 del
< PASS: IN clause with 98000 parameters
22 del
< FAILED QUERY: IN clause with 99000 parameters.
22a19
> FAILED QUERY: IN clause with 97000 parameters.
Test Failed.
*** End:   largeCodeGen jdk1.4.2 DerbyNetClient 2006-05-02 17:08:07 ***

 

Now it looks like it could be a diff problem. Any idea???

 

Thanks, Raman

 

On 2/27/06, John Embretsen <[EMAIL PROTECTED]> wrote:

Kathey Marsden wrote:

> Manjula G Kutty wrote:
>
>> I have closed all the statements, prepared statements and resultsets
>> in the largeCodeGen.java file. But still the test passes with embedded
>> but fails with derbyClient. I'm attaching the modified
>> largeCodeGen.java file
>>
> Hi Manjula,
>
> Because this test throws a lot of exceptions, the statements and
> prepared statements would need to be closed in a finally block.     But
> I just noticed something that might be causing this and it is related to
> the test harness.
> When we run the test network server starts and gets the jvmFlags
> specified for the test in largeCodeGen_app.properties
> jvmflags=-Xmx512M -Xms512M
> (Whether that is at all right I don't know since app_properties is I
> think supposed to be for the client).
>
> But it also gets these conflicting properties (from the harness itself?)
> -ms16777216 -mx33554432
>
> So which is the server really using?

The server is using the latter (-ms16777216 -mx33554432), since these
override the flags passed from the largeCodeGen_app.properties file. I
verified this by running the lang/largeCodeGen.java test from current trunk
with Sun JDK 1.5 and -Dframework=DerbyNetClient.

By the way, in my environment the above test run hangs (using
DerbyNetClient). I have not investigated this further, but the heap does look
full.

I used the jinfo and jmap tools that come with the JDK to observe the flags
received by the server JVM and the actual heap configuration used by the
server while running the test (I got the VMIDs using the jps tool).

By running "jinfo -flags" I got (extract):

VM Flags:
-Xmx512M -Xms512M -Xms16777216 -Xmx33554432
-Dderby.system.home=/home/je159969/apache/Derby/clean/tests/largeCodeGen2/DerbyNetClient/largeCodeGen

-Djava.security.manager
-Djava.security.policy=/home/je159969/apache/Derby/clean/tests/largeCodeGen2/derby_tests.policy

-DderbyTesting.codejar=file:/home/je159969/apache/Derby/clean/trunk/jars/sane/
-DderbyTesting.codedir=/home/je159969/apache/Derby/clean/trunk/jars/sane
-DderbyTesting.serverhost=localhost -DderbyTesting.clienthost=localhost
-DderbyTesting.codeclasses=file://unused/

By running "jmap -heap" I got (extract):

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 33554432 ( 32.0MB)
   NewSize          = 655360 (0.625MB)
   MaxNewSize       = 4294901760 (4095.9375MB)
   OldSize          = 1441792 (1.375MB)
   NewRatio         = 12
   SurvivorRatio    = 32
   PermSize         = 8388608 ( 8.0MB)
   MaxPermSize      = 67108864 (64.0MB)


I.e., jmap reports the max heap size as 32 MB, not 512 MB.

The extra (overriding) jvm heap size flags are set in NetServer.java in the
harness package, which is used by RunTest.java (line 286) to start the
network server. This code seems to be old (and buggy), since it looks for flags
starting with "-ms" or "-mx", not "-Xms" or "-Xmx". Here's the code that is
producing the flags:

if (!jvmName.equals("jview"))
{
    if (setJvmFlags && (jvmflags.indexOf("-ms") == -1))
    // only setMs if no starting memory was given
    jvm.setMs (16*1024*1024); // -ms16m
    if (setJvmFlags && (jvmflags.indexOf("-mx") == -1))
    // only setMx if no max memory was given
    jvm.setMx(32*1024*1024); // -mx32m
    jvm.setNoasyncgc(true); // -noasyncgc
}

The class level comment in jvm.java lists the "valid" jvm options, supposedly
from "java -help". This list does not look anything like the option lists I
get from Sun's JDKs 1.3 through 1.6, so I guess it is a few years old...


--
John

--
Ramandeep Kaur
[EMAIL PROTECTED]