Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/11834 )
Change subject: thirdparty: work around an upstream LLVM build race ...................................................................... Patch Set 2: (2 comments) > Have you observed this succeed with retries? First I tested the loop itself manually (i.e. by replacing the build command with something that fails some of the time), just to make sure the shell scripting was correct. Then I tried to repro the failure with the patch applied, to make sure that it retried properly. I basically ran this in a loop: "rm -rf build/llvm-6.0.0 && PARALLEL=12 ../build-support/enable_devtoolset.sh ./build-thirdparty.sh llvm". I had to add a time.sleep(20) call into the LLVM script that generates the version script, but I did eventually get a repro, and saw that the retry worked: [ 39%] Linking CXX shared library ../../../../lib/clang/6.0.0/lib/linux/libclang_rt.asan-x86_64.so /opt/rh/devtoolset-3/root/usr/libexec/gcc/x86_64-redhat-linux/4.9.2/ld:/data/1/adar/kudu/thirdparty/build/ llvm-6.0.0/projects/compiler-rt/lib/asan/clang_rt.asan-dynamic-x86_64.vers:1: syntax error in VERSION script collect2: error: ld returned 1 exit status make[2]: *** [lib/clang/6.0.0/lib/linux/libclang_rt.asan-x86_64.so] Error 1 make[1]: *** [projects/compiler-rt/lib/asan/CMakeFiles/clang_rt.asan- dynamic-x86_64.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [ 39%] Built target clang_rt.msan_cxx-x86_64-symbols [ 39%] Built target clang_rt.tsan_cxx-x86_64-symbols [ 39%] Built target clang_rt.esan-x86_64-symbols [ 39%] Built target clang_rt.asan-dynamic-x86_64-version-list [ 39%] Built target clang_rt.asan_cxx-x86_64-symbols [ 39%] Built target clang_rt.ubsan_standalone_cxx-x86_64-symbols [ 39%] Built target clang_rt.ubsan_standalone-x86_64-symbols [ 39%] Built target clang_rt.dfsan-x86_64-symbols [ 39%] Built target clang_rt.asan-x86_64-symbols [ 39%] Built target clang_rt.msan-x86_64-symbols [ 39%] Built target clang_rt.ubsan_minimal-x86_64-symbols make: *** [all] Error 2 + exit_status=2 + [[ 2 -eq 0 ]] + [[ 1 -lt 3 ]] + echo 'LLVM build failed, retrying' LLVM build failed, retrying + attempt_number=2 + true + make -j12 install <build goes on to succeed> http://gerrit.cloudera.org:8080/#/c/11834/2/thirdparty/build-definitions.sh File thirdparty/build-definitions.sh: http://gerrit.cloudera.org:8080/#/c/11834/2/thirdparty/build-definitions.sh@347 PS2, Line 347: ${NINJA:-make} -j$PARALLEL $EXTRA_MAKEFLAGS install > Do we need a "clean" before this? i.e., does this actually work if it dies I thought about that, but I don't think we do. When the build fails the version script isn't corrupted so I don't see any reason why we can't piggy-back on the existing build output. http://gerrit.cloudera.org:8080/#/c/11834/2/thirdparty/build-definitions.sh@347 PS2, Line 347: ${NINJA:-make} -j$PARALLEL $EXTRA_MAKEFLAGS install : exit_status=$? : if [[ $exit_status -eq 0 ]]; then : break > This is just: Yes, I had it that way at one point, but this variant, although verbose, is a little more clear if you're less experienced with bash. -- To view, visit http://gerrit.cloudera.org:8080/11834 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3bf2a171b85d34326e187a4d14dbffe7326100d Gerrit-Change-Number: 11834 Gerrit-PatchSet: 2 Gerrit-Owner: Adar Dembo <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Philip Zeyliger <[email protected]> Gerrit-Comment-Date: Wed, 31 Oct 2018 16:31:08 +0000 Gerrit-HasComments: Yes
