IMPALA-7132: Filter out useless output from run_clang_tidy.sh Clang's run-clang-tidy.py script produces a lot of output even when there are no warnings or errors. None of this output is useful.
This patch has two parts: 1. Bump LLVM to 5.0.1-p1, which has patched run-clang-tidy.py to make it reduce its own output when passed -quiet (along with other enhancements). 2. Pass -quiet to run-clang-tidy.py and pipe the stderr output to a temporary file. Display this output only if run-clang-tidy.py hits an error, as this output is not useful otherwise. Testing with a known clang tidy issue shows that warnings and errors are still in the output, and the output is clean on a clean Impala checkout. Change-Id: I63c46a7d57295eba38fac8ab49c7a15d2802df1d Reviewed-on: http://gerrit.cloudera.org:8080/10615 Reviewed-by: Jim Apple <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/impala/repo Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/27c788f8 Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/27c788f8 Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/27c788f8 Branch: refs/heads/master Commit: 27c788f826820791fcdd972b55afd970d6643c1a Parents: b1ec9e3 Author: Joe McDonnell <[email protected]> Authored: Wed Jun 6 12:25:56 2018 -0700 Committer: Jim Apple <[email protected]> Committed: Tue Jul 17 04:58:28 2018 +0000 ---------------------------------------------------------------------- bin/bootstrap_toolchain.py | 2 +- bin/impala-config.sh | 8 ++++---- bin/run_clang_tidy.sh | 10 +++++++++- 3 files changed, 14 insertions(+), 6 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/impala/blob/27c788f8/bin/bootstrap_toolchain.py ---------------------------------------------------------------------- diff --git a/bin/bootstrap_toolchain.py b/bin/bootstrap_toolchain.py index d2a3e71..cbddfbb 100755 --- a/bin/bootstrap_toolchain.py +++ b/bin/bootstrap_toolchain.py @@ -429,7 +429,7 @@ if __name__ == "__main__": "flatbuffers", "gcc", "gflags", "glog", "gperftools", "gtest", "libev", "libunwind", "lz4", "openldap", "openssl", "orc", "protobuf", "rapidjson", "re2", "snappy", "thrift", "tpc-h", "tpc-ds", "zlib"]) - packages.insert(0, Package("llvm", "5.0.1-asserts")) + packages.insert(0, Package("llvm", "5.0.1-asserts-p1")) bootstrap(toolchain_root, packages) # Download the CDH components if necessary. http://git-wip-us.apache.org/repos/asf/impala/blob/27c788f8/bin/impala-config.sh ---------------------------------------------------------------------- diff --git a/bin/impala-config.sh b/bin/impala-config.sh index 0091cd3..d4f40af 100755 --- a/bin/impala-config.sh +++ b/bin/impala-config.sh @@ -68,7 +68,7 @@ fi # moving to a different build of the toolchain, e.g. when a version is bumped or a # compile option is changed. The build id can be found in the output of the toolchain # build jobs, it is constructed from the build number and toolchain git hash prefix. -export IMPALA_TOOLCHAIN_BUILD_ID=137-93cacec18d +export IMPALA_TOOLCHAIN_BUILD_ID=146-f2d5380be6 # Versions of toolchain dependencies. # ----------------------------------- export IMPALA_AVRO_VERSION=1.7.4-p4 @@ -105,14 +105,14 @@ export IMPALA_LIBEV_VERSION=4.20 unset IMPALA_LIBEV_URL export IMPALA_LIBUNWIND_VERSION=1.3-rc1-p3 unset IMPALA_LIBUNWIND_URL -export IMPALA_LLVM_VERSION=5.0.1 +export IMPALA_LLVM_VERSION=5.0.1-p1 unset IMPALA_LLVM_URL -export IMPALA_LLVM_ASAN_VERSION=5.0.1 +export IMPALA_LLVM_ASAN_VERSION=5.0.1-p1 unset IMPALA_LLVM_ASAN_URL # Debug builds should use the release+asserts build to get additional coverage. # Don't use the LLVM debug build because the binaries are too large to distribute. -export IMPALA_LLVM_DEBUG_VERSION=5.0.1-asserts +export IMPALA_LLVM_DEBUG_VERSION=5.0.1-asserts-p1 unset IMPALA_LLVM_DEBUG_URL export IMPALA_LZ4_VERSION=1.7.5 unset IMPALA_LZ4_URL http://git-wip-us.apache.org/repos/asf/impala/blob/27c788f8/bin/run_clang_tidy.sh ---------------------------------------------------------------------- diff --git a/bin/run_clang_tidy.sh b/bin/run_clang_tidy.sh index e879b35..a0e022a 100755 --- a/bin/run_clang_tidy.sh +++ b/bin/run_clang_tidy.sh @@ -54,4 +54,12 @@ fi export PATH="${IMPALA_TOOLCHAIN}/llvm-${IMPALA_LLVM_VERSION}/share/clang\ :${IMPALA_TOOLCHAIN}/llvm-${IMPALA_LLVM_VERSION}/bin/\ :$PATH" -run-clang-tidy.py -header-filter "${PIPE_DIRS%?}" -j"${CORES}" ${DIRS} +TMP_STDERR=$(mktemp) +trap "rm $TMP_STDERR" EXIT +if ! run-clang-tidy.py -quiet -header-filter "${PIPE_DIRS%?}" \ + -j"${CORES}" ${DIRS} 2> ${TMP_STDERR}; +then + echo "run-clang-tidy.py hit an error, dumping stderr output" + cat ${TMP_STDERR} >&2 + exit 1 +fi
