[
https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078749#comment-17078749
]
Michael Stack edited comment on HBASE-23779 at 4/9/20, 4:28 AM:
----------------------------------------------------------------
Branch-2 has the upped nproc patch applied (HBASE-24126) and infra upped the
per-user nproc count from 10000 to 30000. branch-2 also has HBASE-24134 applied
so we should use less memory. The new PR against this issue,
https://github.com/apache/hbase/pull/1461, changes our forkcount from 0.25 to
0.5 and passes -T to mvn of 0.5C. Looking to see the test run time halved.
Below are what branch-2 build times currently look like.
!Screen Shot 2020-04-08 at 9.19.53 AM.png!
was (Author: stack):
Branch-2 has the upped nproc patch applied (HBASE-24126) and infra upped the
per-user nproc count from 10000 to 30000. branch-2 also has HBASE-24134 applied
so we should use less memory. The new PR
https://github.com/apache/hbase/pull/1461 changes our forkcount from 0.25 to
0.5 and passes -T to mvn of 0.5C. Looking to see the test run time halved.
Below are what branch-2 build times currently look like.
!Screen Shot 2020-04-08 at 9.19.53 AM.png!
> Up the default fork count to make builds complete faster; make count relative
> to CPU count
> ------------------------------------------------------------------------------------------
>
> Key: HBASE-23779
> URL: https://issues.apache.org/jira/browse/HBASE-23779
> Project: HBase
> Issue Type: Task
> Components: test
> Reporter: Michael Stack
> Assignee: Michael Stack
> Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: Screen Shot 2020-04-08 at 9.19.53 AM.png,
> addendum2.patch, test_yetus_934.0.patch
>
>
> Tests take a long time. Our fork count running all tests are conservative --
> 1 (small) for first part and 5 for second part (medium and large). Rather
> than hardcoding we should set the fork count to be relative to machine size.
> Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box.
> Looking up at jenkins, it seems like the boxes are 24 cores... at least going
> by my random survey. The load reported on a few seems low though this not
> representative (looking at machine/uptime).
> More parallelism willl probably mean more test failure. Let me take a look
> see.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
