[
https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032548#comment-17032548
]
stack commented on HBASE-23779:
-------------------------------
New addendum. See below. Pushed on Master only for now. Kicked off
https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2/2456/
Lets see. This addendum comes of discussion over in
https://github.com/apache/yetus/pull/86
{code}
commit ef380e0a2e3ba138391968126fd0e9d5c910df88 (HEAD -> m, origin/master,
origin/HEAD)
Author: stack <[email protected]>
Date: Fri Feb 7 09:40:51 2020 -0800
HBASE-23779 Up the default fork count; make count relative to CPU count
(#1108)
ADDENDUM: Refactor that comes of discussion up on
https://github.com/apache/yetus/pull/86
because what I committed originally, and amended in a subsequent
ADDENDUM is not taking effect.
diff --git a/dev-support/hbase-personality.sh b/dev-support/hbase-personality.sh
index 169f19abcc..7f961c0e1e 100755
--- a/dev-support/hbase-personality.sh
+++ b/dev-support/hbase-personality.sh
@@ -82,14 +82,6 @@ function personality_globals
# Override the maven options
MAVEN_OPTS="${MAVEN_OPTS:-"-Xms4G -Xmx4G"}"
- # Pass maven a -T argument. Should make it run faster. Pass conservative
value.
- # Default is one thread. 0.5C on an apache box of 24 cores and 2 executors
should
- # make for 6 threads? Lets see. Setting this here for yetus to pick up. See
- #
https://yetus.apache.org/documentation/0.11.1/precommit-advanced/#global-definitions
- # See below for more on -T:
- #
https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3
- MAVEN_ARGS=("-T0.5C" "${MAVEN_ARGS[@]}")
-
# Yetus 0.7.0 enforces limits. Default proclimit is 1000.
# Up it. See HBASE-19902 for how we arrived at this number.
#shellcheck disable=SC2034
@@ -148,7 +140,11 @@ function personality_modules
clear_personality_queue
- extra="-DHBasePatchProcess"
+ # Pass maven a -T argument. Should make it run faster. Pass conservative
value.
+ # Default is one thread. 0.5C on an apache box of 24 cores and 2 executors
should
+ # make for 6 threads? Lets see. See below for more on -T:
+ #
https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3
+ extra="-T0.5C -DHBasePatchProcess"
if [[ "${PATCH_BRANCH}" = branch-1* ]]; then
extra="${extra} -Dhttps.protocols=TLSv1.2"
fi
{code}
> Up the default fork count to make builds complete faster; make count relative
> to CPU count
> ------------------------------------------------------------------------------------------
>
> Key: HBASE-23779
> URL: https://issues.apache.org/jira/browse/HBASE-23779
> Project: HBase
> Issue Type: Bug
> Components: test
> Reporter: stack
> Assignee: stack
> Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: test_yetus_934.0.patch
>
>
> Tests take a long time. Our fork count running all tests are conservative --
> 1 (small) for first part and 5 for second part (medium and large). Rather
> than hardcoding we should set the fork count to be relative to machine size.
> Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box.
> Looking up at jenkins, it seems like the boxes are 24 cores... at least going
> by my random survey. The load reported on a few seems low though this not
> representative (looking at machine/uptime).
> More parallelism willl probably mean more test failure. Let me take a look
> see.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
