Hello Kudu Jenkins, Adar Dembo,

I'd like you to reexamine a change. Please visit


to look at the new patch set (#2).

Change subject: build: adjust the number of processors for various tests

build: adjust the number of processors for various tests

Many of our tests are multithreaded and can utilize many cores worth of
CPU during execution. Other tests are somewhat timing-sensitive and
might time out if run in coordination with a stress test, particularly
on a small machine which might only have four CPUs total.

This patch adjusts the core count property for a bunch of tests based on
some empirical analysis using a new 'perf-stat' wrapper capability
provided in run-test.sh. The wrapper produces periodic samples of the
CPU usage for each test. Measurements are written into log files which
can then be analyzed with a command like the following:

for x in build/latest/test-logs/*.perf.txt ; do
  echo -n $x
  grep task-clock $x | sort -nk2 | tail -1
done | sort -nk3

This resulted in a list of tests along with their peak CPU usage (from
their busiest 1000ms period). I used this as a rough guide to find tests
with inappropriate settings in CMakeLists.txt. The net effect of this
change is that 'ctest -j' might run slightly longer, but that we'll have
fewer flaky tests in such a run configuration because we don't have
stress tests perturbing timing of unrelated non-stress tests.

Change-Id: I922031af69178b98459d0d59287442b692425afa
M CMakeLists.txt
M build-support/run-test.sh
M src/kudu/cfile/CMakeLists.txt
M src/kudu/client/CMakeLists.txt
M src/kudu/clock/CMakeLists.txt
M src/kudu/consensus/CMakeLists.txt
M src/kudu/integration-tests/CMakeLists.txt
M src/kudu/mini-cluster/CMakeLists.txt
M src/kudu/rpc/CMakeLists.txt
M src/kudu/security/CMakeLists.txt
M src/kudu/tablet/CMakeLists.txt
M src/kudu/tools/CMakeLists.txt
M src/kudu/tserver/CMakeLists.txt
M src/kudu/util/CMakeLists.txt
14 files changed, 79 insertions(+), 47 deletions(-)

  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/03/9503/2
To view, visit http://gerrit.cloudera.org:8080/9503
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I922031af69178b98459d0d59287442b692425afa
Gerrit-Change-Number: 9503
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>

Reply via email to