Adar Dembo has submitted this change and it was merged. Change subject: thirdparty: split into dependency groups ......................................................................
thirdparty: split into dependency groups The monolithic thirdparty build is now quite a bit larger than it used to be on account of the extra LLVM build. Let's see if we can't speed it up. The idea is simple: carve it up into disjoint sections so that individual sections can be rebuilt as needed. This patch separates the various portions of the thirdparty build into "dependency groups". Conceptually, a dependency group is a set of dependencies built a certain way, but the implementation is really just a set of non-overlapping code fragments in build-thirdparty.sh. The initial set of groups are: - common: dependencies that are never instrumented. - uninstrumented: dependencies that may be instrumented, but aren't in this build. - tsan: dependencies that may be instrumented, and are indeed in this build (with -fsanitize=thread). These three generally map to the existing "common", "uninstrumented", and "tsan" thirdparty subdirectories. There's an obvious pattern here for future sanitizer builds (e.g. MSAN would provide an "msan" dependency group). The new build-if-necessary.sh can accept an argument that maps to a set of dependency groups representing a particular build. Every dependency group has its own hash/stamp file so that it is only rebuilt when needed. This also fixes a bug in the stamp file approach that prevented it from actually rebuilding anything. Change-Id: I549262858f98b5ce6c78e786e8c8d8134ba2ed36 Reviewed-on: http://gerrit.cloudera.org:8080/4513 Reviewed-by: Dan Burkert <d...@cloudera.com> Tested-by: Dan Burkert <d...@cloudera.com> --- M CMakeLists.txt M build-support/jenkins/build-and-test.sh M thirdparty/.gitignore M thirdparty/build-if-necessary.sh M thirdparty/build-thirdparty.sh 5 files changed, 271 insertions(+), 202 deletions(-) Approvals: Dan Burkert: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/4513 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I549262858f98b5ce6c78e786e8c8d8134ba2ed36 Gerrit-PatchSet: 5 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Dan Burkert <d...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
