On Mon, 6 Sep 2021 13:22:03 GMT, Aleksey Shipilev <[email protected]> wrote:
>> During the review of JDK-8272914 that added hotspot:tier{2,3} groups,
>> @iignatev suggested to create tier4 groups that capture all tests not in
>> tiers{1,2,3}. I have excluded `vmTestbase` and `hotspot:tier4,` because they
>> take 10+ hours on my highly parallel machine. I have also excluded
>> `applications` from `hotspot:tier4`, because they require test dependencies
>> (e.g. jcstress).
>>
>> Sample run:
>>
>>
>> ==============================
>> Test summary
>> ==============================
>> TEST TOTAL PASS FAIL ERROR
>>
>>>> jtreg:test/hotspot/jtreg:tier4 426 425 1 0
>>>> <<
>>>> jtreg:test/jdk:tier4 2891 2885 4 2
>>>> <<
>> jtreg:test/langtools:tier4 0 0 0 0
>>
>> jtreg:test/jaxp:tier4 0 0 0 0
>>
>> ==============================
>>
>> real 64m13.994s
>> user 1462m1.213s
>> sys 39m38.032s
>>
>>
>> There are interesting test failures on my machine, which I would address
>> separately.
>
> Aleksey Shipilev has updated the pull request incrementally with one
> additional commit since the last revision:
>
> Drop applications and fix the comment
> _Mailing list message from [David Holmes](mailto:[email protected]) on
> [hotspot-dev](mailto:[email protected]):_
>
> On 7/09/2021 1:17 am, Aleksey Shipilev wrote:
>
> > @dholmes-ora: Generally speaking, all `tierX` definitions are rather
> > arbitrary, as there seem to be nothing intrinsic about the tests to be in a
> > particular tier. In other words, what `tierX` consists of is a matter of
> > agreement. I'd say `hotspot:tier4` is "all assorted Hotspot tests that are
> > not application-specific suites"
>
> The difference is that your previous work just consolidated the existing
> subsystem tier 1-3 definitions, but here you are choosing to define "all
> the rest" as tier 4. I don't think it is actually helpful/useful to
> anyone - and it bears no resemblance whatsoever to what we call "tier
> 4", so that will just lead to unnecessary confusion IMO.
@dholmes-ora , although I fully agree that this might lead to some
misunderstanding b/w Oracle and non-Oracle folks, I don't see how it's
different from the previous patch, which introduced `hotspot:tier2` and
`hotspot:tier3`. even if we reduce `tierN` to just a set of tests, the test
groups added by 8272914 bear as much resemblance to the test sets used in
Oracle's tier2-3 as the suggested `hotspot:tier4` groups in this patch to the
actual `tier4` definition used in Oracle's internal system, e.g.
`hotspot:tier2` group has 0 tests from `test/hotspot/jtreg/compiler`, but
Oracle's `tier2` does include a number of `test/hotspot/jtreg/compiler` tests
(which aren't part of `:tier1`). I believe that this patch actually moves us
closer to a convergence point, as the union of `hotspot:tier1` --
`hotspot:tier4` test groups is very close to the test sets used in hotspot
parts of Oracle's `tier1` -- `tier4` definitions.
Thanks,
-- Igor
-------------
PR: https://git.openjdk.java.net/jdk/pull/5357
