When looking at [JDK-8324647](https://bugs.openjdk.org/browse/JDK-8324647), I 
was surprised to see that GHA has this output:


Running test 'jtreg:test/lib-test:tier1'
/home/runner/work/jdk/jdk/test/lib-test/TEST.groups: group all: group includes 
itself
Error: One or more groups are invalid
Finished running test 'jtreg:test/lib-test:tier1'
Test report is stored in 
build/run-test-prebuilt/test-results/jtreg_test_lib_test_tier1

==============================
Test summary
==============================
   TEST TOTAL PASS FAIL ERROR
>> jtreg:test/lib-test:tier1 1 0 0 1 <<
==============================
TEST FAILURE


...and yet, the tests are recorded as "green"! I think this is because our 
error reporting code is tad buggy. AFAICS, `make test-prebuilt` does not report 
a non-zero exit code on failure, and the external script looks for 
`build/run-test-prebuilt/make-support/exit-with-error` to check for error. But 
it does not _set_ the exit code on its own when the failure is discovered, 
neither it sets `failure=true` on generic failure.

I think at very minimum we should report `failure=true` on generic failure. I 
thought about reporting non-zero exit code from the parsing script, but it does 
not feel clean to return non-zero exit code from the _parsing script_. Non-zero 
exit code from that script should signify the error in the script itself.

Additional testing:
 - [x] Current GHA with lib-test failure (now red)
 - [x] Current GHA with lib-test fix (now green)
 - [x] GHA with artificial errors: 
https://github.com/shipilev/jdk/actions/runs/7652797646/job/20854472816

-------------

Commit messages:
 - Merge branch 'master' into JDK-83246590-gha-jtreg-failures
 - Report just the failure=true
 - Also set failure=true on generic error
 - Fix

Changes: https://git.openjdk.org/jdk/pull/17568/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17568&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8324659
  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/17568.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/17568/head:pull/17568

PR: https://git.openjdk.org/jdk/pull/17568

Reply via email to