Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On 23/09/2021 8:33 pm, Thomas Stuefe wrote:
On Thu, 23 Sep 2021 09:19:42 GMT, Aleksey Shipilev wrote:
Now that OpenJDK has more or less complete `tier{1,2,3,4}` definitions, let's
mention them in `testing.md`.
Current patch is my braindump, I am open for suggestions :)
Aleksey Shipilev has updated the pull request incrementally with one additional
commit since the last revision:
More fixes
I like this. Thanks for doing this (and for the actual definition of the
tiers). This hopefully makes communicating about test coverage much easier.
As I understood, tier1-3 are the same as Oracle tier1-3?
Not exactly no. Our tier definitions are tuples of (tests, platforms,
flags). The sets of tests will be close but not exactly the same.
David
-
Marked as reviewed by stuefe (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On Thu, 23 Sep 2021 10:56:02 GMT, Magnus Ihse Bursie wrote: >> Aleksey Shipilev has updated the pull request incrementally with one >> additional commit since the last revision: >> >> More fixes > > doc/testing.md line 95: > >> 93: the tests in `tier1` are carefully selected and optimized to run fast, >> and to run in the >> 94: most stable manner. The test failures in `tier1` are usually followed up >> on quickly, either >> 95: with fixes, or adding relevant tests to problem list. [GitHub Actions >> workflows](../.github/workflows/), > > I don't think the link to the `.github` directory makes sense here. If there > should be a link at all, it should point to information on how GHA works and > how to enable these. And frankly I'm not sure if any such comprehensive > documentation exists. :-( Perhaps in the Skara wiki? Otherwise the link is > just distracting. Sure, I actually put the link there very optimistically :) Dropped it in the new commit. - PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On Thu, 23 Sep 2021 09:19:42 GMT, Aleksey Shipilev wrote:
>> Now that OpenJDK has more or less complete `tier{1,2,3,4}` definitions,
>> let's mention them in `testing.md`.
>>
>> Current patch is my braindump, I am open for suggestions :)
>
> Aleksey Shipilev has updated the pull request incrementally with one
> additional commit since the last revision:
>
> More fixes
I think this looks overall good. Just do something about that GHA link and I'm
perfectly happy! :)
doc/testing.md line 95:
> 93: the tests in `tier1` are carefully selected and optimized to run fast,
> and to run in the
> 94: most stable manner. The test failures in `tier1` are usually followed up
> on quickly, either
> 95: with fixes, or adding relevant tests to problem list. [GitHub Actions
> workflows](../.github/workflows/),
I don't think the link to the `.github` directory makes sense here. If there
should be a link at all, it should point to information on how GHA works and
how to enable these. And frankly I'm not sure if any such comprehensive
documentation exists. :-( Perhaps in the Skara wiki? Otherwise the link is just
distracting.
-
Changes requested by ihse (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On Thu, 23 Sep 2021 09:19:42 GMT, Aleksey Shipilev wrote:
>> Now that OpenJDK has more or less complete `tier{1,2,3,4}` definitions,
>> let's mention them in `testing.md`.
>>
>> Current patch is my braindump, I am open for suggestions :)
>
> Aleksey Shipilev has updated the pull request incrementally with one
> additional commit since the last revision:
>
> More fixes
I like this. Thanks for doing this (and for the actual definition of the
tiers). This hopefully makes communicating about test coverage much easier.
As I understood, tier1-3 are the same as Oracle tier1-3?
-
Marked as reviewed by stuefe (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On Thu, 23 Sep 2021 09:37:49 GMT, Aleksey Shipilev wrote: > No, I think "every other X" puts "X" in singular form: > https://www.merriam-webster.com/dictionary/every%20other. Okay, I agree. Thanks. - PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On Thu, 23 Sep 2021 09:19:42 GMT, Aleksey Shipilev wrote:
>> Now that OpenJDK has more or less complete `tier{1,2,3,4}` definitions,
>> let's mention them in `testing.md`.
>>
>> Current patch is my braindump, I am open for suggestions :)
>
> Aleksey Shipilev has updated the pull request incrementally with one
> additional commit since the last revision:
>
> More fixes
LGTM.
Thanks for this kind of doc update.
-
Marked as reviewed by jiefu (Reviewer).
PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On Thu, 23 Sep 2021 09:28:29 GMT, Jie Fu wrote: >> Aleksey Shipilev has updated the pull request incrementally with one >> additional commit since the last revision: >> >> More fixes > > doc/testing.md line 107: > >> 105: (`JTREG_KEYWORDS=\!headful`), or both. >> 106: >> 107: - `tier4`: This test group includes every other test not covered by >> previous tiers. It includes, > > `other test` => `other tests` No, I think "every other X" puts "X" in singular form: https://www.merriam-webster.com/dictionary/every%20other. - PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
On Thu, 23 Sep 2021 09:19:42 GMT, Aleksey Shipilev wrote:
>> Now that OpenJDK has more or less complete `tier{1,2,3,4}` definitions,
>> let's mention them in `testing.md`.
>>
>> Current patch is my braindump, I am open for suggestions :)
>
> Aleksey Shipilev has updated the pull request incrementally with one
> additional commit since the last revision:
>
> More fixes
doc/testing.md line 107:
> 105: (`JTREG_KEYWORDS=\!headful`), or both.
> 106:
> 107: - `tier4`: This test group includes every other test not covered by
> previous tiers. It includes,
`other test` => `other tests`
-
PR: https://git.openjdk.java.net/jdk/pull/5615
Re: RFR: 8274083: Update testing docs to mention tiered testing [v3]
> Now that OpenJDK has more or less complete `tier{1,2,3,4}` definitions, let's
> mention them in `testing.md`.
>
> Current patch is my braindump, I am open for suggestions :)
Aleksey Shipilev has updated the pull request incrementally with one additional
commit since the last revision:
More fixes
-
Changes:
- all: https://git.openjdk.java.net/jdk/pull/5615/files
- new: https://git.openjdk.java.net/jdk/pull/5615/files/86cbd115..cf2bda57
Webrevs:
- full: https://webrevs.openjdk.java.net/?repo=jdk=5615=02
- incr: https://webrevs.openjdk.java.net/?repo=jdk=5615=01-02
Stats: 7 lines in 2 files changed: 1 ins; 0 del; 6 mod
Patch: https://git.openjdk.java.net/jdk/pull/5615.diff
Fetch: git fetch https://git.openjdk.java.net/jdk pull/5615/head:pull/5615
PR: https://git.openjdk.java.net/jdk/pull/5615
