On Thu, 23 Sep 2021 11:58:02 GMT, Alan Bateman <al...@openjdk.org> wrote:
>> Aleksey Shipilev has updated the pull request incrementally with one >> additional commit since the last revision: >> >> Drop GH link > > doc/testing.html line 79: > >> 77: <ul> >> 78: <li><p><code>tier1</code>: This is the lowest test tier. Multiple >> developers run these tests every day. Normally, at least this tier should be >> clean before integration. Because of the widespread use, the tests in >> <code>tier1</code> are carefully selected and optimized to run fast, and to >> run in the most stable manner. The test failures in <code>tier1</code> are >> usually followed up on quickly, either with fixes, or adding relevant tests >> to problem list. GitHub Actions workflows, if enabled, run >> <code>tier1</code> tests.</p></li> >> 79: <li><p><code>tier2</code>: This test group covers even more ground. >> These contain, among other things, tests that either run for too long to be >> at <code>tier1</code>, tests for less stable and/or experimental features, >> tests for less essential JDK components (for example, jaxp).</p></li> > > I think this a bit unfair on tier2. Tests in tier2 need be run by anyone > making changes in the security area, I/O, networking and many other core > APIs. In general I think anyone changing code in the libraries area needs to > look at TEST.groups to get some idea which tier or finer test group to run. > > I'm not sure about words like "less stable" and "experimental features". Do > we have tests for hotspot experimental features in hotspot_tier2? See new commit. > Tests in tier2 need be run by anyone making changes in the security area, > I/O, networking and many other core APIs. In general I think anyone changing > code in the libraries area needs to look at TEST.groups to get some idea > which tier or finer test group to run. True. That is why the text says that component tests come first, and then you run first N tiers to capture anything else. If any component test is not in the component test group, and only in `tier2`, I'd say that is a test grouping bug, and the test should be in component test group to begin with. In other words, `tier2` testing does not replace the component group testing, and thus the requirement to run `tier2` if particular areas are changed would feel awkward to me. > Do we have tests for hotspot experimental features in hotspot_tier2? We do move deprecating things out of `tier1` (see e.g. recent JDK-8272838). ------------- PR: https://git.openjdk.java.net/jdk/pull/5615
