Todd Lipcon has submitted this change and it was merged. ( )

Change subject: build: adjust the number of processors for various tests

build: adjust the number of processors for various tests

Many of our tests are multithreaded and can utilize many cores worth of
CPU during execution. Other tests are somewhat timing-sensitive and
might time out if run in coordination with a stress test, particularly
on a small machine which might only have four CPUs total.

This patch adjusts the core count property for a bunch of tests based on
some empirical analysis using a new 'perf-stat' wrapper capability
provided in The wrapper produces periodic samples of the
CPU usage for each test. Measurements are written into log files which
can then be analyzed with a command like the following:

for x in build/latest/test-logs/*.perf.txt ; do
  echo -n $x
  grep task-clock $x | sort -nk2 | tail -1
done | sort -nk3

This resulted in a list of tests along with their peak CPU usage (from
their busiest 1000ms period). I used this as a rough guide to find tests
with inappropriate settings in CMakeLists.txt. The net effect of this
change is that 'ctest -j' might run slightly longer, but that we'll have
fewer flaky tests in such a run configuration because we don't have
stress tests perturbing timing of unrelated non-stress tests.

Change-Id: I922031af69178b98459d0d59287442b692425afa
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <>
M CMakeLists.txt
M build-support/
M src/kudu/cfile/CMakeLists.txt
M src/kudu/client/CMakeLists.txt
M src/kudu/clock/CMakeLists.txt
M src/kudu/consensus/CMakeLists.txt
M src/kudu/integration-tests/CMakeLists.txt
M src/kudu/mini-cluster/CMakeLists.txt
M src/kudu/rpc/CMakeLists.txt
M src/kudu/security/CMakeLists.txt
M src/kudu/tablet/CMakeLists.txt
M src/kudu/tools/CMakeLists.txt
M src/kudu/tserver/CMakeLists.txt
M src/kudu/util/CMakeLists.txt
14 files changed, 80 insertions(+), 48 deletions(-)

  Kudu Jenkins: Verified
  Adar Dembo: Looks good to me, approved

To view, visit
To unsubscribe, visit

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I922031af69178b98459d0d59287442b692425afa
Gerrit-Change-Number: 9503
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <>
Gerrit-Reviewer: Adar Dembo <>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <>

Reply via email to