Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Erik Joelsson]

Looks good, thanks for cleaning up the whole file!

/Erik

On 2020-06-24 15:39, Magnus Ihse Bursie wrote:

On 2020-06-15 23:23, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.
Ok, I have now added the new options to the documentation. I double 
checked to see that all other options were documented. Almost all 
were, but not FAILURE_HANDLER_TIMEOUT, so I added that as well.


When looking into testing.md I noticed that it had diverged quite far 
from our normal standards of code quality, so I made some adjustments 
to get it back to consistency, including fixing line length and adding 
empty lines after headings. I also fixed the "keword" typo that Jon 
noticed.


There are no other changes except for testing.md/.html in this webrev:

http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.02 



/Magnus


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg documentation, 
such as a new page describing these options for the JDK makefile 
support, giving due deference to the makefile (like "If in doubt, 
consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly 
to the java that launches jtreg, something that was needed, 
especially to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options 
to the launcher. Also, for the specific case of changing the max 
output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed 
while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Magnus Ihse Bursie]

On 2020-06-15 23:23, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.
Ok, I have now added the new options to the documentation. I double 
checked to see that all other options were documented. Almost all were, 
but not FAILURE_HANDLER_TIMEOUT, so I added that as well.


When looking into testing.md I noticed that it had diverged quite far 
from our normal standards of code quality, so I made some adjustments to 
get it back to consistency, including fixing line length and adding 
empty lines after headings. I also fixed the "keword" typo that Jon noticed.


There are no other changes except for testing.md/.html in this webrev:

http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.02

/Magnus


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg documentation, 
such as a new page describing these options for the JDK makefile 
support, giving due deference to the makefile (like "If in doubt, 
consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly to 
the java that launches jtreg, something that was needed, especially 
to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options 
to the launcher. Also, for the specific case of changing the max 
output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT VERBOSE 
RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT OPTIONS 
JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS AOT_MODULES 
LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed 
while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Magnus Ihse Bursie]

On 2020-06-16 16:45, Magnus Ihse Bursie wrote:

On 2020-06-16 16:35, Jonathan Gibbons wrote:



On 6/16/20 5:26 AM, Magnus Ihse Bursie wrote:

On 2020-06-16 04:33, Jonathan Gibbons wrote:

Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: 
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html


Actually, that link is going to get stale in just a few months time. 
Probably better to link here directly:


https://github.com/openjdk/jdk/blob/master/doc/testing.md
The end of that file, as presented in a browser, does not look like 
it is as you maybe intended.

Yes. Skip that. Use this instead:
https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk/master/doc/testing.html 


Actually, use this instead:
http://openjdk.java.net/groups/build/doc/testing.html

I have finally managed to fix so the Build Group web page has a 
perma-link to the latest updated build and test instructions. It should 
have been done long ago... But now it's done!


For the record, the perma-link to the build readme is 
http://openjdk.java.net/groups/build/doc/building.html. I will make a 
separate effort to try and promote this link, and remove all other links.


/Magnus



/Magnus


The heading is split and the following text is shown literally:


header-includes:

  * 'pre, code, tt { color: #1d6ae5; }'

Maybe it's a pandoc-ism, but it makes it less than ideal to link to 
the file on github.


-- Jon



/Magnus



-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg 
documentation, such as a new page describing these options for 
the JDK makefile support, giving due deference to the makefile 
(like "If in doubt, consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be 
a good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments 
directly to the java that launches jtreg, something that was 
needed, especially to change the output limit using 
Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic 
options to the launcher. Also, for the specific case of changing 
the max output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added 
a help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I 
noticed while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

test defaults: was Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Jonathan Gibbons]

On defaults for running jtreg tests ...

In my experience, and given the separate recent discussion with Stefan 
Karlsson,
there are two significant modalities for running tests: running lots of 
tests,
and just running a single test, typically while developing or debugging 
the test.


With that in mind, in the personal scripts I use to run jtreg for my own JDK
development work, I determine whether the tests to be run are a single test
or multiple tests, and then adjust the defaults accordingly. Defaults 
that you

might want to change are
  * the verbose level
  * the retain option

Maybe it might be interesting to do something similar in the run-test 
farmework.


-- Jon




On 6/16/20 7:45 AM, Magnus Ihse Bursie wrote:

On 2020-06-16 16:35, Jonathan Gibbons wrote:



On 6/16/20 5:26 AM, Magnus Ihse Bursie wrote:

On 2020-06-16 04:33, Jonathan Gibbons wrote:

Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: 
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html


Actually, that link is going to get stale in just a few months time. 
Probably better to link here directly:


https://github.com/openjdk/jdk/blob/master/doc/testing.md
The end of that file, as presented in a browser, does not look like 
it is as you maybe intended.

Yes. Skip that. Use this instead:
https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk/master/doc/testing.html

/Magnus


The heading is split and the following text is shown literally:


header-includes:

  * 'pre, code, tt { color: #1d6ae5; }'

Maybe it's a pandoc-ism, but it makes it less than ideal to link to 
the file on github.


-- Jon



/Magnus



-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg 
documentation, such as a new page describing these options for 
the JDK makefile support, giving due deference to the makefile 
(like "If in doubt, consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be 
a good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments 
directly to the java that launches jtreg, something that was 
needed, especially to change the output limit using 
Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic 
options to the launcher. Also, for the specific case of changing 
the max output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added 
a help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I 
noticed while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Jonathan Gibbons]

Magnus,

There's a mild ambiguity in the use of the phrase "JTReg keywords" which 
is used to describe both the mechanism in the run-test framework, and 
the jtreg test selection mechanism.


That aside, while reading the file, I noticed the following typo (kewords):


   KEYWORDS

JTReg kewords


-- Jon


On 6/16/20 7:45 AM, Magnus Ihse Bursie wrote:

On 2020-06-16 16:35, Jonathan Gibbons wrote:



On 6/16/20 5:26 AM, Magnus Ihse Bursie wrote:

On 2020-06-16 04:33, Jonathan Gibbons wrote:

Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: 
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html


Actually, that link is going to get stale in just a few months time. 
Probably better to link here directly:


https://github.com/openjdk/jdk/blob/master/doc/testing.md
The end of that file, as presented in a browser, does not look like 
it is as you maybe intended.

Yes. Skip that. Use this instead:
https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk/master/doc/testing.html

/Magnus


The heading is split and the following text is shown literally:


header-includes:

  * 'pre, code, tt { color: #1d6ae5; }'

Maybe it's a pandoc-ism, but it makes it less than ideal to link to 
the file on github.


-- Jon



/Magnus



-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg 
documentation, such as a new page describing these options for 
the JDK makefile support, giving due deference to the makefile 
(like "If in doubt, consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be 
a good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments 
directly to the java that launches jtreg, something that was 
needed, especially to change the output limit using 
Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic 
options to the launcher. Also, for the specific case of changing 
the max output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added 
a help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I 
noticed while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Magnus Ihse Bursie]

On 2020-06-16 16:35, Jonathan Gibbons wrote:



On 6/16/20 5:26 AM, Magnus Ihse Bursie wrote:

On 2020-06-16 04:33, Jonathan Gibbons wrote:

Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: 
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html


Actually, that link is going to get stale in just a few months time. 
Probably better to link here directly:


https://github.com/openjdk/jdk/blob/master/doc/testing.md
The end of that file, as presented in a browser, does not look like it 
is as you maybe intended.

Yes. Skip that. Use this instead:
https://htmlpreview.github.io/?https://raw.githubusercontent.com/openjdk/jdk/master/doc/testing.html

/Magnus


The heading is split and the following text is shown literally:


header-includes:

  * 'pre, code, tt { color: #1d6ae5; }'

Maybe it's a pandoc-ism, but it makes it less than ideal to link to 
the file on github.


-- Jon



/Magnus



-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg 
documentation, such as a new page describing these options for the 
JDK makefile support, giving due deference to the makefile (like 
"If in doubt, consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly 
to the java that launches jtreg, something that was needed, 
especially to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic 
options to the launcher. Also, for the specific case of changing 
the max output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I 
noticed while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Jonathan Gibbons]

On 6/16/20 5:26 AM, Magnus Ihse Bursie wrote:

On 2020-06-16 04:33, Jonathan Gibbons wrote:

Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: 
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html


Actually, that link is going to get stale in just a few months time. 
Probably better to link here directly:


https://github.com/openjdk/jdk/blob/master/doc/testing.md
The end of that file, as presented in a browser, does not look like it 
is as you maybe intended.

The heading is split and the following text is shown literally:


header-includes:

 * 'pre, code, tt { color: #1d6ae5; }'

Maybe it's a pandoc-ism, but it makes it less than ideal to link to the 
file on github.


-- Jon



/Magnus



-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg 
documentation, such as a new page describing these options for the 
JDK makefile support, giving due deference to the makefile (like 
"If in doubt, consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly 
to the java that launches jtreg, something that was needed, 
especially to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options 
to the launcher. Also, for the specific case of changing the max 
output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed 
while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Magnus Ihse Bursie]

On 2020-06-16 14:31, Lance Andersen wrote:

Hi Magnus,

Can we update this page to include an example of how to run a JTREG 
manual test via the make infrastructure?
For the record: After some off-list discussion, Lance opened 
https://bugs.openjdk.java.net/browse/JDK-8247690 for supporting manual 
test runs.


/Magnus



Best
Lance

On Jun 16, 2020, at 8:26 AM, Magnus Ihse Bursie 
> wrote:


On 2020-06-16 04:33, Jonathan Gibbons wrote:

Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: 
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html


Actually, that link is going to get stale in just a few months time. 
Probably better to link here directly:


https://github.com/openjdk/jdk/blob/master/doc/testing.md

/Magnus



-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg 
documentation, such as a new page describing these options for the 
JDK makefile support, giving due deference to the makefile (like 
"If in doubt, consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly 
to the java that launches jtreg, something that was needed, 
especially to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic 
options to the launcher. Also, for the specific case of changing 
the max output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I 
noticed while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus






Lance 
Andersen| Principal Member of Technical Staff | +1.781.442.2037

Oracle Java Engineering
1 Network Drive
Burlington, MA 01803
lance.ander...@oracle.com 

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Lance Andersen]

Hi Magnus,

Can we update this page to include an example of how to run a JTREG manual test 
via the make infrastructure?

Best
Lance

> On Jun 16, 2020, at 8:26 AM, Magnus Ihse Bursie 
>  wrote:
> 
> On 2020-06-16 04:33, Jonathan Gibbons wrote:
>> Thanks; I'll link to that page from the jtreg FAQ.
> ... but please use the tip URL: 
> https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html
> 
> Actually, that link is going to get stale in just a few months time. Probably 
> better to link here directly:
> 
> https://github.com/openjdk/jdk/blob/master/doc/testing.md
> 
> /Magnus
> 
>> 
>> -- Jon
>> 
>> On 6/15/20 2:23 PM, Erik Joelsson wrote:
>>> The makefile test documentation is in the repo, so you could link directly 
>>> to it:
>>> 
>>> https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 
>>> 
>>> Which reminds me that Magnus should update that documentation with these 
>>> new options.
>>> 
>>> /Erik
>>> 
>>> On 2020-06-15 14:19, Jonathan Gibbons wrote:
 Magnus,
 
 Would it be worth while adding something to the jtreg documentation, such 
 as a new page describing these options for the JDK makefile support, 
 giving due deference to the makefile (like "If in doubt, consult `make 
 test JTREG=help`")
 
 I can write the page, with your help, if you think this would be a good 
 idea.
 
 -- Jon
 
 On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
> It turned out that it was not possible to send arguments directly to the 
> java that launches jtreg, something that was needed, especially to change 
> the output limit using Djavatest.maxOutputSize.
> 
> This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options to the 
> launcher. Also, for the specific case of changing the max output size, it 
> adds JTREG=MAX_OUTPUT=nnn.
> 
> All these new options made me realize there's no way apart from reading 
> the code to know what options are available, so I added a help system, 
> and documented this in "make help". Now you can get:
> 
> $ make test JTREG=help
> Building target 'test' in configuration 'linux-x64'
> Valid keywords for JTREG:
> JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT VERBOSE 
> RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT OPTIONS 
> JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS AOT_MODULES 
> LAUNCHER_OPTIONS.
> RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.
> 
> Finally, I fixed some broken comments regarding AOT that I noticed while 
> fixing this.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01
> 
> /Magnus
> 

 
  

 Lance Andersen| 
Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
lance.ander...@oracle.com 

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Magnus Ihse Bursie]

On 2020-06-16 04:33, Jonathan Gibbons wrote:

Thanks; I'll link to that page from the jtreg FAQ.
... but please use the tip URL: 
https://hg.openjdk.java.net/jdk/jdk/raw-file/tip/doc/testing.html


Actually, that link is going to get stale in just a few months time. 
Probably better to link here directly:


https://github.com/openjdk/jdk/blob/master/doc/testing.md

/Magnus



-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg documentation, 
such as a new page describing these options for the JDK makefile 
support, giving due deference to the makefile (like "If in doubt, 
consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly 
to the java that launches jtreg, something that was needed, 
especially to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options 
to the launcher. Also, for the specific case of changing the max 
output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT 
VERBOSE RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT 
OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS 
AOT_MODULES LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed 
while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Jonathan Gibbons]

Thanks; I'll link to that page from the jtreg FAQ.

-- Jon

On 6/15/20 2:23 PM, Erik Joelsson wrote:
The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html 



Which reminds me that Magnus should update that documentation with 
these new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg documentation, 
such as a new page describing these options for the JDK makefile 
support, giving due deference to the makefile (like "If in doubt, 
consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly to 
the java that launches jtreg, something that was needed, especially 
to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options 
to the launcher. Also, for the specific case of changing the max 
output size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT VERBOSE 
RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT OPTIONS 
JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS AOT_MODULES 
LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed 
while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Re: Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Erik Joelsson]

The makefile test documentation is in the repo, so you could link 
directly to it:


https://hg.openjdk.java.net/jdk/jdk/raw-file/87828d2e32fe/doc/testing.html

Which reminds me that Magnus should update that documentation with these 
new options.


/Erik

On 2020-06-15 14:19, Jonathan Gibbons wrote:

Magnus,

Would it be worth while adding something to the jtreg documentation, 
such as a new page describing these options for the JDK makefile 
support, giving due deference to the makefile (like "If in doubt, 
consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a 
good idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly to 
the java that launches jtreg, something that was needed, especially 
to change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options to 
the launcher. Also, for the specific case of changing the max output 
size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a 
help system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT VERBOSE 
RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT OPTIONS 
JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS AOT_MODULES 
LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed 
while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus

Documenting options; was Re: RFR: JDK-8220639 Need a way to augment JTREG_LAUNCHER_OPTIONS from command-line

[Jonathan Gibbons]

Magnus,

Would it be worth while adding something to the jtreg documentation, 
such as a new page describing these options for the JDK makefile 
support, giving due deference to the makefile (like "If in doubt, 
consult `make test JTREG=help`")


I can write the page, with your help, if you think this would be a good 
idea.


-- Jon

On 6/15/20 8:56 AM, Magnus Ihse Bursie wrote:
It turned out that it was not possible to send arguments directly to 
the java that launches jtreg, something that was needed, especially to 
change the output limit using Djavatest.maxOutputSize.


This patch adds JTREG=LAUNCHER_OPTIONS=xxx to pass generic options to 
the launcher. Also, for the specific case of changing the max output 
size, it adds JTREG=MAX_OUTPUT=nnn.


All these new options made me realize there's no way apart from 
reading the code to know what options are available, so I added a help 
system, and documented this in "make help". Now you can get:


$ make test JTREG=help
Building target 'test' in configuration 'linux-x64'
Valid keywords for JTREG:
JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT TEST_MODE ASSERT VERBOSE 
RETAIN MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT MAX_OUTPUT OPTIONS 
JAVA_OPTIONS VM_OPTIONS KEYWORDS EXTRA_PROBLEM_LISTS AOT_MODULES 
LAUNCHER_OPTIONS.

RunTests.gmk:292: *** Re-run without 'help' to continue. Stop.

Finally, I fixed some broken comments regarding AOT that I noticed 
while fixing this.


Bug: https://bugs.openjdk.java.net/browse/JDK-8220639
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8220639-add-jtreg-launcher-options/webrev.01


/Magnus