This adds the test repeat feature in the build system. This is convenient to follow-up on intermittently failing tests: the `REPEAT_COUNT > 0` would run the test multiple times, until we run out of repeats or the tests fail.
With this sample test: /* * @test * @run driver IntermittentTest */ public class IntermittentTest { public static void main(String... args) { if (Math.random() < 0.05) { throw new IllegalStateException("Oops"); } } } ...a lucky run looks like this: $ CONF=linux-x86_64-server-fastdebug make run-test TEST=sanity/IntermittentTest.java JTREG="REPEAT_COUNT=5" Building target 'run-test' in configuration 'linux-x86_64-server-fastdebug' Running tests using JTREG control variable 'REPEAT_COUNT=5' Test selection 'sanity/IntermittentTest.java', will run: * jtreg:test/hotspot/jtreg/sanity/IntermittentTest.java Running test 'jtreg:test/hotspot/jtreg/sanity/IntermittentTest.java' Repeating Jtreg run: 1 out of 5 Passed: sanity/IntermittentTest.java Test results: passed: 1 Report written to ... Results written to ... Repeating Jtreg run: 2 out of 5 Passed: sanity/IntermittentTest.java Test results: passed: 1 Report written to ... Results written to ... Repeating Jtreg run: 3 out of 5 Passed: sanity/IntermittentTest.java Test results: passed: 1 Report written to ... Results written to ... Repeating Jtreg run: 4 out of 5 Passed: sanity/IntermittentTest.java Test results: passed: 1 Report written to ... Results written to ... Repeating Jtreg run: 5 out of 5 Passed: sanity/IntermittentTest.java Test results: passed: 1 Report written to ... Results written to ... Finished running test 'jtreg:test/hotspot/jtreg/sanity/IntermittentTest.java' Test report is stored in ... ============================== Test summary ============================== TEST TOTAL PASS FAIL ERROR jtreg:test/hotspot/jtreg/sanity/IntermittentTest.java 1 1 0 0 ============================== TEST SUCCESS ...and unlucky run looks like this: $ CONF=linux-x86_64-server-fastdebug make run-test TEST=sanity/IntermittentTest.java JTREG="REPEAT_COUNT=50" Building target 'run-test' in configuration 'linux-x86_64-server-fastdebug' Running tests using JTREG control variable 'REPEAT_COUNT=50' Test selection 'sanity/IntermittentTest.java', will run: * jtreg:test/hotspot/jtreg/sanity/IntermittentTest.java ... Repeating Jtreg run: 9 out of 50 Passed: sanity/IntermittentTest.java Test results: passed: 1 Report written to ... Results written to ... Repeating Jtreg run: 10 out of 50 -------------------------------------------------- TEST: sanity/IntermittentTest.java TEST JDK: ... ... STDERR: java.lang.IllegalStateException: Oops at IntermittentTest.main(IntermittentTest.java:8) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) at java.base/java.lang.reflect.Method.invoke(Method.java:577) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:312) at java.base/java.lang.Thread.run(Thread.java:833) JavaTest Message: Test threw exception: java.lang.IllegalStateException JavaTest Message: shutting down test TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.IllegalStateException: Oops -------------------------------------------------- Test results: failed: 1 Report written to ... Results written to ... Error: Some tests failed or other problems occurred. Failures detected, no more repeats. Finished running test 'jtreg:test/hotspot/jtreg/sanity/IntermittentTest.java' Test report is stored in ... ============================== Test summary ============================== TEST TOTAL PASS FAIL ERROR jtreg:test/hotspot/jtreg/sanity/IntermittentTest.java >> 1 0 1 0 << ============================== TEST FAILURE ------------- Commit messages: - Fix Changes: https://git.openjdk.java.net/jdk/pull/6720/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6720&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8244602 Stats: 28 lines in 3 files changed: 27 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6720.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6720/head:pull/6720 PR: https://git.openjdk.java.net/jdk/pull/6720