[
https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027649#comment-17027649
]
Michael Stack commented on HBASE-23779:
---------------------------------------
Chatting w/ Sean out on slack:
{code}
saintstack Yesterday at 3:57 PM
Looking at faster builds (prompted by a colleagues' exploration). Was looking
to up the default fork count. Our jenkins builds seem to use default. Was going
to do a fraction of CPUs. Looking at random build, there are 24 CPUs in the
build boxes. Looking at the uptime load, says '2'. Could look around more but
was thinking of going from default of 4 forks to 0.5CPU#. Testing locally to
make sure pass profile is not distrubted (Our test classification has gotten
messy... cleaning that up too). Shout if input. Thanks.
6 replies
busbey 16 hours ago
Remember there are 2 executors per node, so optimistically we get 12 cores
busbey 15 hours ago
And if we land on a node with Hadoop tests then maybe only like half that
saintstack 3 minutes ago
So, 0.5CPU sounds like a good number then as its pretty much what we have now
saintstack 3 minutes ago
... only it has advantage of varying w/ machine cpu count.
saintstack 2 minutes ago
...rather than being hardcoded.
{code}
... so let me down what I have here from .75C to 0.5C so we are close to the
settings we already have (The patch has 0.75 -- not 0.5 as I suggest in the
comment).
> Up the default fork count to make builds complete faster; make count relative
> to CPU count
> ------------------------------------------------------------------------------------------
>
> Key: HBASE-23779
> URL: https://issues.apache.org/jira/browse/HBASE-23779
> Project: HBase
> Issue Type: Bug
> Components: test
> Reporter: Michael Stack
> Priority: Major
>
> Tests take a long time. Our fork count running all tests are conservative --
> 1 (small) for first part and 5 for second part (medium and large). Rather
> than hardcoding we should set the fork count to be relative to machine size.
> Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box.
> Looking up at jenkins, it seems like the boxes are 24 cores... at least going
> by my random survey. The load reported on a few seems low though this not
> representative (looking at machine/uptime).
> More parallelism willl probably mean more test failure. Let me take a look
> see.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
