On defaults for running jtreg tests ...
In my experience, and given the separate recent discussion with Stefan
Karlsson,
there are two significant modalities for running tests: running lots of
tests,
and just running a single test, typically while developing or debugging
the test.
With that in mind, in the personal scripts I use to run jtreg for my own JDK
development work, I determine whether the tests to be run are a single test
or multiple tests, and then adjust the defaults accordingly. Defaults
that you
might want to change are
* the verbose level
* the retain option
Maybe it might be interesting to do something similar in the run-test
farmework.
-- Jon
On 6/16/20 7:45 AM, Magnus Ihse Bursie wrote:
On 2020-06-16 16:35, Jonathan Gibbons wrote:
On 6/16/20 5:26 AM, Magnus Ihse Bursie wrote:
On 2020-06-16 04:33, Jonathan Gibbons wrote:
Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL:
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html
Actually, that link is going to get stale in just a few months time.
Probably better to link here directly:
https://github.com/openjdk/jdk/blob/master/doc/testing.md
The end of that file, as presented in a browser, does not look like
it is as you maybe intended.
Yes. Skip that. Use this instead:
https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk/master/doc/testing.html
/Magnus
The heading is split and the following text is shown literally:
header-includes:
* '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>'
Maybe it's a pandoc-ism, but it makes it less than ideal to link to
the file on github.
-- Jon
/Magnus
-- Jon
On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link
directly to it:
https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html
Which reminds me that Magnus should update that documentation with
these new options.
/Erik
On 2020-06-15 14:19, Jonathan Gibbons wrote:
Magnus,
Would it be worth while adding something to the jtreg
documentation, such as a new page describing these options for
the JDK makefile support, giving due deference to the makefile
(like "If in doubt, consult `make test JTREG=help`")
I can write the page, with your help, if you think this would be
a good idea.
-- Jon
On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments
directly to the java that launches jtreg, something that was
needed, especially to change the output limit using
Djavatest.maxOutputSize.
This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic
options to the launcher. Also, for the specific case of changing
the max output size, it adds JTREG=MAX_OUTPUT=nnn.
All these new options made me realize there's no way apart from
reading the code to know what options are available, so I added
a help system, and documented this in "make help". Now you can get:
$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS
AOT_MODULES LAUNCHER_OPTIONS.
RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.
Finally, I fixed some broken comments regarding AOT that I
noticed while fixing this.
Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev:
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01
/Magnus
