On defaults for running jtreg tests ...

In my experience, and given the separate recent discussion with Stefan Karlsson, there are two significant modalities for running tests: running lots of tests, and just running a single test, typically while developing or debugging the test.

With that in mind, in the personal scripts I use to run jtreg for my own JDK
development work, I determine whether the tests to be run are a single test
or multiple tests, and then adjust the defaults accordingly. Defaults that you
might want to change are
  * the verbose level
  * the retain option

Maybe it might be interesting to do something similar in the run-test farmework.

-- Jon




On 6/16/20 7:45 AM, Magnus Ihse Bursie wrote:
On 2020-06-16 16:35, Jonathan Gibbons wrote:


On 6/16/20 5:26 AM, Magnus Ihse Bursie wrote:
On 2020-06-16 04:33, Jonathan Gibbons wrote:
Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html

Actually, that link is going to get stale in just a few months time. Probably better to link here directly:

https://github.com/openjdk/jdk/blob/master/doc/testing.md
The end of that file, as presented in a browser, does not look like it is as you maybe intended.
Yes. Skip that. Use this instead:
https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk/master/doc/testing.html

/Magnus

The heading is split and the following text is shown literally:


header-includes:

  * '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>'

Maybe it's a pandoc-ism, but it makes it less than ideal to link to the file on github.

-- Jon


/Magnus


-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link directly to it:

https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html

Which reminds me that Magnus should update that documentation with these new options.

/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:
Magnus,

Would it be worth while adding something to the jtreg documentation, such as a new page describing these options for the JDK makefile support, giving due deference to the makefile (like "If in doubt, consult `make test JTREG=help`")

I can write the page, with your help, if you think this would be a good idea.

-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly to the java that launches jtreg, something that was needed, especially to change the output limit using Djavatest.maxOutputSize.

This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options to the launcher. Also, for the specific case of changing the max output size, it adds JTREG=MAX_OUTPUT=nnn.

All these new options made me realize there's no way apart from reading the code to know what options are available, so I added a help system, and documented this in "make help". Now you can get:

$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS AOT_MODULES LAUNCHER_OPTIONS.
RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed while fixing this.

Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01

/Magnus


Reply via email to