[
https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23779.
-----------------------------------
Release Note:
Passing --threads=2 when building on jenkins shortens nightly build times by
about ~25%. It works by running module build/test in parallel when dependencies
allow. Upping the forkcount beyond the pom default of 0.25C would have us
broach our CPU budget up on jenkins when two modules are running in parallel;
it also seems to threaten the build stability.
For running tests locally, to go faster:
$ x="0.5C" ; mvn -T2 -Dsurefire.firstPartForkCount=$x
-Dsurefire.secondPartForkCount=$x test -PrunAllTests
You could up the x from 0.5C to 1.0C but YMMV (On overcommitted hardware, tests
start bombing out pretty soon after startup).
Resolution: Fixed
> Up the default fork count to make builds complete faster; make count relative
> to CPU count
> ------------------------------------------------------------------------------------------
>
> Key: HBASE-23779
> URL: https://issues.apache.org/jira/browse/HBASE-23779
> Project: HBase
> Issue Type: Task
> Components: test
> Reporter: Michael Stack
> Assignee: Michael Stack
> Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: Screen Shot 2020-04-08 at 9.19.53 AM.png, Screen Shot
> 2020-04-13 at 4.09.49 PM.png, addendum2.patch, test_yetus_934.0.patch
>
>
> Tests take a long time. Our fork count running all tests are conservative --
> 1 (small) for first part and 5 for second part (medium and large). Rather
> than hardcoding we should set the fork count to be relative to machine size.
> Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box.
> Looking up at jenkins, it seems like the boxes are 24 cores... at least going
> by my random survey. The load reported on a few seems low though this not
> representative (looking at machine/uptime).
> More parallelism willl probably mean more test failure. Let me take a look
> see.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
