Hi Leonid,
The exclusion of these 3 tests from tier1 looks ok to me. It did lead me
to some other questions however. The first was that if you exclude them
from tier1, how to they get included in a later tier. The answer, for
the SA tests, is that all of serviceability is included in tier2. So the
next question was whether or not we want to repeat testing in higher
tiers. I suppose since higher tiers include new platforms and build
options, the answer is yes, but I hope the upper tiers are constructed
well enough that we aren't wasting too much time repeating the same
test(s) on the same platforms with the same options.
thanks,
Chris
On 3/26/18 4:32 PM, Leonid Mesnik wrote:
Hi
There is no strict rules and time budgets for hotspot tiers. It is assumed that
faster tests should be executed earlier tiers. The order and total impact of of
these test depends on how they are executed.
Here is time of execution of tier1 groups with and without these excluded tests:
The whole :tier1 time reduced from 40 to 30 minutes (on dedicated HW with 32
core)
Time for :tier1_serviceability reduced from 15 to 6 min (in HS CI)
Time for :tier1_gc_1 reduced from 12 to 8-10 min (in HS CI)
The benefits of exclusion of gc/g1/ihop/TestIHOPErgo.java are not so
significant. However it is the only one GC stress test executed in tier1 while
intention is to don’t run stress testing in tier1.
Tests
serviceability/sa/ClhsdbScanOops.java
serviceability/sa/TestHeapDumpForLargeArray.java
are now hotspot_tier3_runtime which includes all hotspot_serviceability tests
which are not a part of tier1. They are executed as a part of tier3 now as well
as most of serviceability tests.
Test gc/g1/ihop/TestIHOPErgo.java in now in group :hotspot_gc only and also it
is marked as “stress” test. So it is executed with all other GC stress tests
Leonid
On Mar 25, 2018, at 3:48 PM, David Holmes <[email protected]> wrote:
Hi Leonid,
On 24/03/2018 9:31 AM, Leonid Mesnik wrote:
Hi
Could you please review following fix which exclude following tests from tier1
testing:
serviceability/sa/ClhsdbScanOops.java
serviceability/sa/TestHeapDumpForLargeArray.java
gc/g1/ihop/TestIHOPErgo.java
Each of them takes more then 5 minutes to complete and significantly increase
overall time to complete tier1.
I'd need to see a much more detailed analysis of all the tests run, the order
they run and the execution times to ascertain what impact this actually has on
overall test execution time.
But assuming 5 minutes is too long for tier1, this seems okay. But the tests
must still be run in some other tier, and I'm not clear where that would be
now? I would expect them to move to tier 3 perhaps, depending on what the time
criteria for tier 3 is.
Thanks,
David
Please let me know if there are any reasons to run these tests in tier1 despite
on their execution time.
webrev:http://cr.openjdk.java.net/~lmesnik/8200187/webrev.00/
bug: https://bugs.openjdk.java.net/browse/JDK-8200187
Leonid
