Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
Hi Andrew, On Fri, 2019-06-28 at 14:45 +0100, Andrew John Hughes wrote: > On 28/06/2019 10:52, Severin Gehwolf wrote: > > Hi Andrew, > > > > On Thu, 2019-06-27 at 17:36 +0100, Andrew John Hughes wrote: > > > On 22/05/2019 17:34, Severin Gehwolf wrote: > > > > Hi, > > > > > > > > Could I please get reviews for this minimal implementation of a tier1- > > > > like test set for JDK 8u? The implementation is rather barebones as I > > > > don't think it's worth rewriting the build system just for a command > > > > that runs a certain set of tests across a select set of repositories. > > > > I've re-used existing work in Makefiles as much as possible. After this > > > > patch one can do: > > > > > > > > $ make test TEST="tier1" > > > > > > > > Inspiration came from JDK 11u's tier1. As for prior art to this, I've > > > > only found "make test" to be working for JDK 8u from the top level. > > > > Yet, it doesn't run any hotspot tests, exits with a zero code on test > > > > failures and doesn't present a summary at the end. Overall not a nice > > > > developer experience. > > > > > > > > This patch makes it easier for a developers tests. It presents a > > > > summary at the end, returns non-zero on test failures so this can get > > > > used in CI and runs hotspot tests. > > > > > > > > As a follow-on we can work on fixing/excluding tests so that we always > > > > have a passing set of tests for developers to run before a checkin. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > > > > (includes changes to top/hotspot/jdk/langtools repos) > > > > > > > > Example excerpt from a run: > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt > > > > > > > > Thoughts? > > > > > > > > Thanks, > > > > Severin > > > > > > > > > > Is there a reason for creating new tier definitions here rather than > > > backporting the existing ones? > > > > Yes. The tests in JDK 8 are significantly different from JDK 11. Most > > notably the number of tests available and defined test groups. > > Backporting the JDK 11 changes seems like fitting a square peg into a > > round hole. > > > > The JDK 8u "tier1" could only be a rough approximation of the JDK 11 > > "tier1" at best. There is also no intention to make the test output > > 100% identical to JDK 11 as bringing back those changes would be too > > invasive. > > > > I believe the proposed patch is a reasonable minimal patch which aids > > 8u developer's testing. > > > > > https://bugs.openjdk.java.net/browse/JDK-8075543 > > > > > > The subtasks also cover nashorn & jaxp which are missed here. jaxp would > > > need JDK-8065673, JDK-8051540 and friends to convert its tests to jtreg. > > > > Sure. We can consider adding later tiers in upcoming work. That's not > > the priority, though. Note that tier1 tests in JDK 11 only run > > jdk/lantools/hotspot tests. The intention of this initial patch would > > be to have some testing "baseline" covering a reasonable set of repos > > and tests. > > > > Thanks, > > Severin > > > > I'm not talking about JDK 11 changes. These are changes in JDK 9. > > For example, > > https://hg.openjdk.java.net/jdk9/dev/jdk/rev/cd4aea326e89 > > adds groups to two tiers, all of which exist in 8u as well. It's not > clear where the list in your patch comes from and it appears to add a > bunch of arbitrary other tests beyond the tier 1 defined in the JDK 9 > change. Other changes to the JDK tier1 definition were taken from current JDK 11u. > It would make more sense to me to backport JDK-8075544 & JDK-8075573 as > pre-requisites rather than creating completely new definitions. Done now. I've added fix-request comments/labels to the above bugs and rebased on top of them. New jdk changeset: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/04/jdk/webrev/ Test groups definition is the JDK 9 set plus :jdk_jdi test set (part of JDK-8198551 in later JDKs). This seems a reasonable test set. Modulo added intrinsics testing only relevant for 9+, see JDK-8132855 and JDK- 8132854. OK to push? Thanks, Severin
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
On 28/06/2019 10:52, Severin Gehwolf wrote: > Hi Andrew, > > On Thu, 2019-06-27 at 17:36 +0100, Andrew John Hughes wrote: >> >> On 22/05/2019 17:34, Severin Gehwolf wrote: >>> Hi, >>> >>> Could I please get reviews for this minimal implementation of a tier1- >>> like test set for JDK 8u? The implementation is rather barebones as I >>> don't think it's worth rewriting the build system just for a command >>> that runs a certain set of tests across a select set of repositories. >>> I've re-used existing work in Makefiles as much as possible. After this >>> patch one can do: >>> >>> $ make test TEST="tier1" >>> >>> Inspiration came from JDK 11u's tier1. As for prior art to this, I've >>> only found "make test" to be working for JDK 8u from the top level. >>> Yet, it doesn't run any hotspot tests, exits with a zero code on test >>> failures and doesn't present a summary at the end. Overall not a nice >>> developer experience. >>> >>> This patch makes it easier for a developers tests. It presents a >>> summary at the end, returns non-zero on test failures so this can get >>> used in CI and runs hotspot tests. >>> >>> As a follow-on we can work on fixing/excluding tests so that we always >>> have a passing set of tests for developers to run before a checkin. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 >>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ >>> (includes changes to top/hotspot/jdk/langtools repos) >>> >>> Example excerpt from a run: >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt >>> >>> Thoughts? >>> >>> Thanks, >>> Severin >>> >> >> Is there a reason for creating new tier definitions here rather than >> backporting the existing ones? > > Yes. The tests in JDK 8 are significantly different from JDK 11. Most > notably the number of tests available and defined test groups. > Backporting the JDK 11 changes seems like fitting a square peg into a > round hole. > > The JDK 8u "tier1" could only be a rough approximation of the JDK 11 > "tier1" at best. There is also no intention to make the test output > 100% identical to JDK 11 as bringing back those changes would be too > invasive. > > I believe the proposed patch is a reasonable minimal patch which aids > 8u developer's testing. > >> https://bugs.openjdk.java.net/browse/JDK-8075543 >> >> The subtasks also cover nashorn & jaxp which are missed here. jaxp would >> need JDK-8065673, JDK-8051540 and friends to convert its tests to jtreg. > > Sure. We can consider adding later tiers in upcoming work. That's not > the priority, though. Note that tier1 tests in JDK 11 only run > jdk/lantools/hotspot tests. The intention of this initial patch would > be to have some testing "baseline" covering a reasonable set of repos > and tests. > > Thanks, > Severin > I'm not talking about JDK 11 changes. These are changes in JDK 9. For example, https://hg.openjdk.java.net/jdk9/dev/jdk/rev/cd4aea326e89 adds groups to two tiers, all of which exist in 8u as well. It's not clear where the list in your patch comes from and it appears to add a bunch of arbitrary other tests beyond the tier 1 defined in the JDK 9 change. It would make more sense to me to backport JDK-8075544 & JDK-8075573 as pre-requisites rather than creating completely new definitions. -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 https://keybase.io/gnu_andrew
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
Hi Andrew, On Thu, 2019-06-27 at 17:36 +0100, Andrew John Hughes wrote: > > On 22/05/2019 17:34, Severin Gehwolf wrote: > > Hi, > > > > Could I please get reviews for this minimal implementation of a tier1- > > like test set for JDK 8u? The implementation is rather barebones as I > > don't think it's worth rewriting the build system just for a command > > that runs a certain set of tests across a select set of repositories. > > I've re-used existing work in Makefiles as much as possible. After this > > patch one can do: > > > > $ make test TEST="tier1" > > > > Inspiration came from JDK 11u's tier1. As for prior art to this, I've > > only found "make test" to be working for JDK 8u from the top level. > > Yet, it doesn't run any hotspot tests, exits with a zero code on test > > failures and doesn't present a summary at the end. Overall not a nice > > developer experience. > > > > This patch makes it easier for a developers tests. It presents a > > summary at the end, returns non-zero on test failures so this can get > > used in CI and runs hotspot tests. > > > > As a follow-on we can work on fixing/excluding tests so that we always > > have a passing set of tests for developers to run before a checkin. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > > (includes changes to top/hotspot/jdk/langtools repos) > > > > Example excerpt from a run: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt > > > > Thoughts? > > > > Thanks, > > Severin > > > > Is there a reason for creating new tier definitions here rather than > backporting the existing ones? Yes. The tests in JDK 8 are significantly different from JDK 11. Most notably the number of tests available and defined test groups. Backporting the JDK 11 changes seems like fitting a square peg into a round hole. The JDK 8u "tier1" could only be a rough approximation of the JDK 11 "tier1" at best. There is also no intention to make the test output 100% identical to JDK 11 as bringing back those changes would be too invasive. I believe the proposed patch is a reasonable minimal patch which aids 8u developer's testing. > https://bugs.openjdk.java.net/browse/JDK-8075543 > > The subtasks also cover nashorn & jaxp which are missed here. jaxp would > need JDK-8065673, JDK-8051540 and friends to convert its tests to jtreg. Sure. We can consider adding later tiers in upcoming work. That's not the priority, though. Note that tier1 tests in JDK 11 only run jdk/lantools/hotspot tests. The intention of this initial patch would be to have some testing "baseline" covering a reasonable set of repos and tests. Thanks, Severin
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
On 22/05/2019 17:34, Severin Gehwolf wrote: > Hi, > > Could I please get reviews for this minimal implementation of a tier1- > like test set for JDK 8u? The implementation is rather barebones as I > don't think it's worth rewriting the build system just for a command > that runs a certain set of tests across a select set of repositories. > I've re-used existing work in Makefiles as much as possible. After this > patch one can do: > > $ make test TEST="tier1" > > Inspiration came from JDK 11u's tier1. As for prior art to this, I've > only found "make test" to be working for JDK 8u from the top level. > Yet, it doesn't run any hotspot tests, exits with a zero code on test > failures and doesn't present a summary at the end. Overall not a nice > developer experience. > > This patch makes it easier for a developers tests. It presents a > summary at the end, returns non-zero on test failures so this can get > used in CI and runs hotspot tests. > > As a follow-on we can work on fixing/excluding tests so that we always > have a passing set of tests for developers to run before a checkin. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > (includes changes to top/hotspot/jdk/langtools repos) > > Example excerpt from a run: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt > > Thoughts? > > Thanks, > Severin > Is there a reason for creating new tier definitions here rather than backporting the existing ones? https://bugs.openjdk.java.net/browse/JDK-8075543 The subtasks also cover nashorn & jaxp which are missed here. jaxp would need JDK-8065673, JDK-8051540 and friends to convert its tests to jtreg. -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 https://keybase.io/gnu_andrew
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
On 6/27/19 1:50 PM, Severin Gehwolf wrote: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/03/langtools/webrev/ > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/03/hotspot/webrev/ Looks fine to me. -- Thanks, -Aleksey
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
On Thu, 2019-06-27 at 13:27 +0200, Aleksey Shipilev wrote: > On 6/27/19 1:20 PM, Severin Gehwolf wrote: > > On Thu, 2019-06-27 at 11:58 +0200, Aleksey Shipilev wrote: > > > On 5/22/19 6:34 PM, Severin Gehwolf wrote: > > > hotspot: > > > > > > *) Test exclusion question. Do we really want to exclude these tests from > > > tier1? I would rather see > > > them included and make them fail in regular runs. This would put pressure > > > on fixing them for 8u... > > > > > > 150 # Right now tier1 runs all hotspot tests except RTM tests, > > > 151 # and one serviceability test which seems flakey. > > > 152 hotspot_tier1 = \ > > > 153 :jdk \ > > > 154 -compiler/rtm \ > > > 155 -serviceability/sa/jmap-hashcode/Test8028623.java > > > > I'm not sure about it. We could exclude nothing and go from there. > > However, the more tests we have failing from the get-go the less useful > > it is to enforce tier1 tests to always pass for JDK 8u (bad > > signal/noise ratio). I'd be OK for Test8028623.java, but the > > compiler/rtm tests, I'm not so sure... Thoughts? > > From my perspective, new test profile would inevitably show some (new) > failures. > > We would need to work to fix those failures before we can rely on low SNR in > tier1. Ignoring tests > may skip over the tests that need attention during that transitional period. > We don't ignore the > tests that are failing right now in fastdebug, right? Same thing here. > > In other words, I would rather see tier1 suite to land, and then work on > fixing/ignoring tests once > we understand how/if they are fixable. OK. Updated langtools/hotspot webrevs with no exclusions. jdk had no exclusions to begin with: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/03/langtools/webrev/ http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/03/hotspot/webrev/ Thoughts? Thanks, Severin
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
On 6/27/19 1:20 PM, Severin Gehwolf wrote: > On Thu, 2019-06-27 at 11:58 +0200, Aleksey Shipilev wrote: >> On 5/22/19 6:34 PM, Severin Gehwolf wrote: >> hotspot: >> >> *) Test exclusion question. Do we really want to exclude these tests from >> tier1? I would rather see >> them included and make them fail in regular runs. This would put pressure on >> fixing them for 8u... >> >> 150 # Right now tier1 runs all hotspot tests except RTM tests, >> 151 # and one serviceability test which seems flakey. >> 152 hotspot_tier1 = \ >> 153 :jdk \ >> 154 -compiler/rtm \ >> 155 -serviceability/sa/jmap-hashcode/Test8028623.java > > I'm not sure about it. We could exclude nothing and go from there. > However, the more tests we have failing from the get-go the less useful > it is to enforce tier1 tests to always pass for JDK 8u (bad > signal/noise ratio). I'd be OK for Test8028623.java, but the > compiler/rtm tests, I'm not so sure... Thoughts? From my perspective, new test profile would inevitably show some (new) failures. We would need to work to fix those failures before we can rely on low SNR in tier1. Ignoring tests may skip over the tests that need attention during that transitional period. We don't ignore the tests that are failing right now in fastdebug, right? Same thing here. In other words, I would rather see tier1 suite to land, and then work on fixing/ignoring tests once we understand how/if they are fixable. -- Thanks, -Aleksey
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
On Thu, 2019-06-27 at 11:58 +0200, Aleksey Shipilev wrote: > On 5/22/19 6:34 PM, Severin Gehwolf wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > > (includes changes to top/hotspot/jdk/langtools repos) > > Excellent work! I eyeballed makefile changes and they seem fine (not an > expert, though). Thanks for the review, Aleksey! > hotspot: > > *) Test exclusion question. Do we really want to exclude these tests from > tier1? I would rather see > them included and make them fail in regular runs. This would put pressure on > fixing them for 8u... > > 150 # Right now tier1 runs all hotspot tests except RTM tests, > 151 # and one serviceability test which seems flakey. > 152 hotspot_tier1 = \ > 153 :jdk \ > 154 -compiler/rtm \ > 155 -serviceability/sa/jmap-hashcode/Test8028623.java I'm not sure about it. We could exclude nothing and go from there. However, the more tests we have failing from the get-go the less useful it is to enforce tier1 tests to always pass for JDK 8u (bad signal/noise ratio). I'd be OK for Test8028623.java, but the compiler/rtm tests, I'm not so sure... Thoughts? > Ran it on my current jdk8u-dev: > > $ CONF=linux-x86_64-normal-server-fastdebug make images test TEST=tier1 > > Summary: langtools_tier1 > FAILED: tools/javac/lambda/LambdaParserTest.java > TEST STATS: name=langtools_tier1 run=3113 pass=3112 fail=1 > > Summary: hotspot_tier1 > FAILED: compiler/ciReplay/TestSA.sh > FAILED: compiler/intrinsics/bmi/verifycode/TZcntTestL.java > FAILED: gc/arguments/TestAggressiveHeap.java > FAILED: runtime/RedefineObject/TestRedefineObject.java > FAILED: runtime/StackGap/testme.sh > TEST STATS: name=hotspot_tier1 run=729 pass=724 fail=5 > > Summary: jdk_tier1 > TEST STATS: name=jdk_tier1 run=1379 pass=1379 fail=0 This seems about right for fastdebug builds. > $ echo $? > 2 > > So, summary is there, exit code is right, and some tests are failing (which > is good for testing this > patch!). Any suggestions on how to proceed with this? Do you want me to do a v3 with no hotspot tests excluded? It would be nice if I could get this patch off of my queue ;-) Thanks, Severin
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
On 5/22/19 6:34 PM, Severin Gehwolf wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > (includes changes to top/hotspot/jdk/langtools repos) Excellent work! I eyeballed makefile changes and they seem fine (not an expert, though). hotspot: *) Test exclusion question. Do we really want to exclude these tests from tier1? I would rather see them included and make them fail in regular runs. This would put pressure on fixing them for 8u... 150 # Right now tier1 runs all hotspot tests except RTM tests, 151 # and one serviceability test which seems flakey. 152 hotspot_tier1 = \ 153 :jdk \ 154 -compiler/rtm \ 155 -serviceability/sa/jmap-hashcode/Test8028623.java Ran it on my current jdk8u-dev: $ CONF=linux-x86_64-normal-server-fastdebug make images test TEST=tier1 Summary: langtools_tier1 FAILED: tools/javac/lambda/LambdaParserTest.java TEST STATS: name=langtools_tier1 run=3113 pass=3112 fail=1 Summary: hotspot_tier1 FAILED: compiler/ciReplay/TestSA.sh FAILED: compiler/intrinsics/bmi/verifycode/TZcntTestL.java FAILED: gc/arguments/TestAggressiveHeap.java FAILED: runtime/RedefineObject/TestRedefineObject.java FAILED: runtime/StackGap/testme.sh TEST STATS: name=hotspot_tier1 run=729 pass=724 fail=5 Summary: jdk_tier1 TEST STATS: name=jdk_tier1 run=1379 pass=1379 fail=0 $ echo $? 2 So, summary is there, exit code is right, and some tests are failing (which is good for testing this patch!). -- Thanks, -Aleksey
PING? Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
Hi! Any more thoughts on this? It would be nice to get this reviewed :) Thanks, Severin On Thu, 2019-05-23 at 11:53 +0200, Severin Gehwolf wrote: > Hi Andrew, > > On Thu, 2019-05-23 at 09:34 +0100, Andrew Dinn wrote: > > Hi Severin, > > > > On 22/05/2019 17:34, Severin Gehwolf wrote: > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > > > (includes changes to top/hotspot/jdk/langtools repos) > > > > > > Example excerpt from a run: > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt > > > > > > Thoughts? > > Unfortunately, my makefile fu is too limited to provide a reliable > > review of the patch (I'm mostly just surprised it took so few changes to > > achieve). However, I can still wholeheartedly applaud this endeavour and > > its present outcome. > > :) Great. > > > As you say tuning the test group can be done over time as a follow-up. > > The important thing is to quickly place devs in a position to easily > > check that fixes have no unintended side-effects. Very nice! > > Thanks for looking this over! > > Cheers, > Severin
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
Hi Andrew, On Thu, 2019-05-23 at 09:34 +0100, Andrew Dinn wrote: > Hi Severin, > > On 22/05/2019 17:34, Severin Gehwolf wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > > (includes changes to top/hotspot/jdk/langtools repos) > > > > Example excerpt from a run: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt > > > > Thoughts? > Unfortunately, my makefile fu is too limited to provide a reliable > review of the patch (I'm mostly just surprised it took so few changes to > achieve). However, I can still wholeheartedly applaud this endeavour and > its present outcome. :) Great. > As you say tuning the test group can be done over time as a follow-up. > The important thing is to quickly place devs in a position to easily > check that fixes have no unintended side-effects. Very nice! Thanks for looking this over! Cheers, Severin
Re: [8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
Hi Severin,
On 22/05/2019 17:34, Severin Gehwolf wrote:
> Bug: https://bugs.openjdk.java.net/browse/JDK-8222737
> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/
> (includes changes to top/hotspot/jdk/langtools repos)
>
> Example excerpt from a run:
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt
>
> Thoughts?
Unfortunately, my makefile fu is too limited to provide a reliable
review of the patch (I'm mostly just surprised it took so few changes to
achieve). However, I can still wholeheartedly applaud this endeavour and
its present outcome.
As you say tuning the test group can be done over time as a follow-up.
The important thing is to quickly place devs in a position to easily
check that fixes have no unintended side-effects. Very nice!
regards,
Andrew Dinn
---
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
[8u] RFR: 8222737: [TESTBUG] Allow for tier 1 like testing in OpenJDK 8u
Hi, Could I please get reviews for this minimal implementation of a tier1- like test set for JDK 8u? The implementation is rather barebones as I don't think it's worth rewriting the build system just for a command that runs a certain set of tests across a select set of repositories. I've re-used existing work in Makefiles as much as possible. After this patch one can do: $ make test TEST="tier1" Inspiration came from JDK 11u's tier1. As for prior art to this, I've only found "make test" to be working for JDK 8u from the top level. Yet, it doesn't run any hotspot tests, exits with a zero code on test failures and doesn't present a summary at the end. Overall not a nice developer experience. This patch makes it easier for a developers tests. It presents a summary at the end, returns non-zero on test failures so this can get used in CI and runs hotspot tests. As a follow-on we can work on fixing/excluding tests so that we always have a passing set of tests for developers to run before a checkin. Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ (includes changes to top/hotspot/jdk/langtools repos) Example excerpt from a run: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt Thoughts? Thanks, Severin
