[
https://issues.apache.org/jira/browse/HBASE-23780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23780.
-----------------------------------
Fix Version/s: master
2.3.0
Hadoop Flags: Reviewed
Assignee: Michael Stack
Resolution: Fixed
Pushed this on branch-2 and master. Reclassifies tests according to how long
they take in recent runs.
Idea is to move as many tests as possible into small category where they all
run in the one JVM. Allows us to move through a bunch in a short time.
Next up is getting tests into the medium category. These are tests that take <
50 seconds. Parallelization should help here as they don't stick around too
long. TODO is fix tests that only run a single test and the test takes little
time... Aggregate up single tests into one test suite. Medium tests start a
JVM. This costs. Amortize.
Large tests we'll just have to tackle one at a time.
I've been using a coworker's script to up the parallelization:
{code}$ x=1.0 mvn -T $xC -Dsurefire.firstPartForkCount=$xC
-Dsurefire.secondPartForkCount=$xC -PrunSmallTests{code} ... etc.
The -T in particular helps by making maven use more than one thread.
> Edit of test classifications
> ----------------------------
>
> Key: HBASE-23780
> URL: https://issues.apache.org/jira/browse/HBASE-23780
> Project: HBase
> Issue Type: Bug
> Components: test
> Reporter: Michael Stack
> Assignee: Michael Stack
> Priority: Major
> Fix For: 2.3.0, master
>
>
> Our test classifications have drifted. You can see for yourself running each
> of the small/medium and large test suites. See test complete times. See how
> even some large tests should be small and vice versa.
> The more small tests we can run inside the single JVM, the faster we'll get
> through the build. Tests that are Medium start their own JVM for each test.
> Tests that are Medium but only last a second or two are expensive and should
> be aggregated with other single, short tests to amortize the JVM startup.
> Anyways, let me edit the test categories and try and clean them up some.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
