[
https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033283#comment-17033283
]
Mark Robert Miller commented on HBASE-23779:
--------------------------------------------
Couple notes I've noticed:
# -T doesn't not seem to work so well for downloading deps at the least. If I
don't do a non -T to download and make sure I have all deps at some point
first, I see crashes.
# In my experience, cannot create native thread due to OOM tends to be a RAM
issue more often than open file limit issue. You can often help that by not
accepting default huge stack sizes per thread - you don't often need nearly as
much as some of the high defaults for that these days - try a megabyte.
# More threads and tests at the same time uses more RAM of course, another way
to help is to peg an Xms of like 256 or something, rather than just setting Xmx
- encourage the tests that don't need so much RAM to not claim it to begin
with.
> Up the default fork count to make builds complete faster; make count relative
> to CPU count
> ------------------------------------------------------------------------------------------
>
> Key: HBASE-23779
> URL: https://issues.apache.org/jira/browse/HBASE-23779
> Project: HBase
> Issue Type: Bug
> Components: test
> Reporter: Michael Stack
> Assignee: Michael Stack
> Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: addendum2.patch, test_yetus_934.0.patch
>
>
> Tests take a long time. Our fork count running all tests are conservative --
> 1 (small) for first part and 5 for second part (medium and large). Rather
> than hardcoding we should set the fork count to be relative to machine size.
> Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box.
> Looking up at jenkins, it seems like the boxes are 24 cores... at least going
> by my random survey. The load reported on a few seems low though this not
> representative (looking at machine/uptime).
> More parallelism willl probably mean more test failure. Let me take a look
> see.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
