On 14/11/2014 08:40, Staffan Larsen wrote:
:
So the goal here has been to increase the test coverage of hotspot
jprt push jobs, but with a limited impact on execution time. This is
all to make sure hotspot changes do no break serviceability features.
While it would be great to run all tests at all times, we don’t have
time for that. Mikael has been doing code coverage analysis to find
the subset of test that provides the biggest bang for the buck.
Starting with JDI is as good as any place to start.
I agree that listing individual tests is not particularly appealing,
but I don’t see many other options. We could possibly use @key tags to
select the tests but there isn’t much support in makefiles and jprt
for that if I recall. We could use sub-folders, but that quickly gets
out of hand.
We could move the _sanity lists to one place in the file to make it
easier to see the rest.
My main concern is keeping the test group hierarchy easy to understand
and maintain. It has to be easy to identify any tests that aren't run by
any of the top-level groups for example. For serviceability then the
original idea was to have the jdk_svc group that ran all of the tests,
this in turn consisted of 5 sub-groups to cover the various areas. This
sub-groups will execute concurrently on different JPRT clients and works
reasonably well, albeit with some imbalance in the execution time.
The "real" definitions of the groups end just after the section with the
comment "Client area groups" so if you can move the new sanity groups
down to below that with a good comment to explain what they are then it
should okay.
At some point we need to look at removing completely the "Profile based
..." groups at the end. They are completely unmaintainable and there are
much better ways of doing this now (with @requires).
-Alan
