On 11/12/2013 17:55, Jonathan Gibbons wrote:
Balchandra,
The important part of that of interest here is the part about "calls
jtreg commands
(one each for jdk, langtools and hotspot) with those recommended
options".
Is there a way you could post here the segment of the script that does
that?
Here it is
2.1 Running tests in jdk/test
$ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary
-exclude:{openjdk source top directory}/jdk/test/ProblemList.txt
-conc:auto -a -ignore:quiet -timeoutFactor:5 -othervm -testjdk:{location
of the test jdk} `cat dir.list
<http://download.java.net/jdk8/testresults/docs/dir.list>`
2.2 Running tests in langtools/test
$ jtreg -dir:{openjdk source top directory}/langtools/test
-verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm
-testjdk:{location of the test jdk} com tools
2.3 Running tests in hotspot/test
$ jtreg -dir:{openjdk source top directory}/hotspot/test
-verbose:summary -conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm
-testjdk:{location of the test jdk} compiler gc runtime sanity
serviceability
The instructions is at
http://download.java.net/jdk8/testresults/docs/howtoruntests.html
and is linked from
http://www.java.net/download/jdk8/testresults/testresults.html
The caveat in this approach is a human error where one (me) forget to
update the instruction
or forget to remove any additional flags used in the script temporarily
- a mismatch
of the results could occur.
From the next build (b120), I am going to update "2.1 Running tests in
jdk/test" to
$ jtreg -dir:{openjdk source top directory}/jdk/test -verbose:summary
-exclude:{openjdk source top directory}/jdk/test/ProblemList.txt
-conc:auto -a -ignore:quiet -timeoutFactor:5 -agentvm -testjdk:{location
of the test jdk} :jdk_core :jdk_svc :jdk_beans :jdk_imageio :jdk_sound
:jdk_sctp javax/accessibility com/sun/java/swing javax/print sun/pisces
com/sun/awt
Alternatively, that segment of the script could be a candidate for a
target in
one or more test/Makefile files.
This is good idea, but my experience with the 'make' is that if one
target critically fail, all
subsequent targets will not run. I thought it is a restriction of 'make'.
Thanks
Balchandra
-- Jon
On 12/11/2013 08:58 AM, Balchandra Vaidya wrote:
Hi Volker,
The custom script I use just wrap the instruction posted (except
building jtreg). It just clones the openjdk repository, calls jtreg
commands
(one each for jdk, langtools and hotspot) with those recommended
options, runs diffs, runs nashorn tests, and archives results.
Such convenient script may not fit under openjdk repo, but think of
adding it in code-tools project in the future.
Thanks
Balchandra
On 11/12/2013 13:45, Volker Simonis wrote:
Hi Balchandra,
what do you think about checking in a little script or Makefile target
which produces the test results you report for the ea builds?
This would be of great help to produce comparable test results.
Regards,
Volker
On Tue, Dec 10, 2013 at 5:39 PM, Balchandra Vaidya
<balchandra.vai...@oracle.com> wrote:
On 12/10/13 03:25 PM, Alan Bateman wrote:
On 10/12/2013 11:20, Balchandra Vaidya wrote:
:
2) Testcase do not run with -agentvm option. It might be an
issue in the testcase (or how the testcase was run). I will
look into
it.
I missed this issue because I had changed my scripts to use
-othervm
sometime back for debugging/analyzing some testcase
failure but forgot to change it back. Thank you for pointing
it out.
For Volker's benefit, othervm is where jtreg spins up a new VM for
each
test, agentvm is where jtreg re-uses the VM if possible. There are
tests
that don't or can't clean up and those tests need to ensure that
their @run
tests have the /othervm option so that the tests runs in its own VM.
Alternatively, the TEST.ROOT file has a key that lists directories
where all
the tests in those directory trees must run in their own VM.
Balchandra - one thing that be useful is to look at the client
tests to
see what can run in agentvm and what can't. For the jdk_core and
jdk_svc
tests then they can be run in either (although we're still picking
off
issues as they found in agentvm mode). I'm not aware of any work
done yet on
getting the tests in the jdk_desktop test group to run in agentvm
mode. It
may be that TEST.ROOT needs to be updated to see a few additional
top-level
directories so that all the tests in these areas temporarily run
in othervm
mode.
In the subset of the tests I have been running, it appears only one
test [1]
is failing with agentvm option. I will try to run other tests in
client
groups [Note: I will update the instruction to use jtreg groups
with b119
results posting] and see what tests can be run in agentvm mode, but
main
issue
in those remaining client tests are some tests may not consistently
produce
same results - could stabilizing those tests be a goal for jdk9 repo ?
Thanks
Balchandra
[1] com/sun/java/swing/plaf/windows/8016551/bug8016551.java
-Alan
