Hello Dan Burkert, Todd Lipcon,

I'd like you to do a code review.  Please visit


to review the following change.

Change subject: thirdparty: split into dependency groups

thirdparty: split into dependency groups

The monolithic thirdparty build is now quite a bit larger than it used to be
on account of the extra LLVM build. Let's see if we can't speed it up. The
idea is simple: carve it up into disjoint sections so that individual
sections can be rebuilt as needed.

This patch separates the various portions of the thirdparty build into
"dependency groups". Conceptually, a dependency group is a set of
dependencies built a certain way, but the implementation is really just a
set of non-overlapping code fragments in build-thirdparty.sh.

The initial set of groups are:
- common: dependencies that are never instrumented.
- uninstrumented: dependencies that may be instrumented, but aren't in this
- instrumented_tsan: dependencies that may be instrumented, and are indeed
  in this build (with -fsanitize=thread).

These three generally map to the existing "installed", "installed-deps", and
"installed-deps-tsan" thirdparty subdirectories. There's an obvious pattern
here for future sanitizer builds (e.g. MSAN would provide an
"instrumented_msan" dependency group).

The new build-if-necessary.sh can accept an argument that maps to a set of
dependency groups representing a particular build. Every dependency group
has its own hash/stamp file so that it is only rebuilt when needed.

This also fixes a bug in the stamp file approach that prevented it from
actually rebuilding anything.

Change-Id: I549262858f98b5ce6c78e786e8c8d8134ba2ed36
M CMakeLists.txt
M build-support/jenkins/build-and-test.sh
M thirdparty/.gitignore
M thirdparty/build-if-necessary.sh
M thirdparty/build-thirdparty.sh
5 files changed, 256 insertions(+), 195 deletions(-)

  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/4513/1
To view, visit http://gerrit.cloudera.org:8080/4513
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I549262858f98b5ce6c78e786e8c8d8134ba2ed36
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <d...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>

Reply via email to