This is an automated email from the ASF dual-hosted git repository. joemcdonnell pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/impala.git
commit 4c582fc55be3028e9a42ad527c9b2678d6106faf Author: Joe McDonnell <[email protected]> AuthorDate: Wed Sep 18 19:19:33 2024 -0700 IMPALA-12686: Switch to toolchain with basic debug info This switches to a toolchain that has been built with basic debug information (-g1). This is useful for getting better stack traces when in library code. The toolchain has also been built with -gz to compress the debug information. Some components already built with more debug information (e.g. -g) and the new toolchain preserves this. This skips adding debug information for tools like CMake, Mold, etc. It also skips adding debug information for LLVM's release build. Even at -g1, LLVM's release build has an enormous amount of debug information, and it would add hundreds of MBs to impalad's binary size to include it. This adds about 31MB to the compressed binary size for Impala. It actually reduces the size of the toolchain by a few hundred MB due to the compression. However, all libraries now have more debug information than they did before. Link commands use a bit more memory than before. The final build in build-all-flag-combinations.sh tests setting a custom version for the Java build. Everything is in ccache at that point, so if it builds the backend tests, there will be many link invocations running simultaneously, which can overload the system memory. This modifies that location to use -notests, as it is not testing the build of backend tests. Testing: - Ran core tests - Checked for changes in build time Change-Id: I7b962c350cc5f1f2b24ca7a52b940ec9e87a7745 Reviewed-on: http://gerrit.cloudera.org:8080/21471 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Michael Smith <[email protected]> Reviewed-by: Laszlo Gaal <[email protected]> --- bin/impala-config.sh | 6 +++--- bin/jenkins/build-all-flag-combinations.sh | 6 +++++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/bin/impala-config.sh b/bin/impala-config.sh index 09b2fba48..336a1631c 100755 --- a/bin/impala-config.sh +++ b/bin/impala-config.sh @@ -81,13 +81,13 @@ export USE_AVRO_CPP=${USE_AVRO_CPP:=false} # moving to a different build of the toolchain, e.g. when a version is bumped or a # compile option is changed. The build id can be found in the output of the toolchain # build jobs, it is constructed from the build number and toolchain git hash prefix. -export IMPALA_TOOLCHAIN_BUILD_ID_AARCH64=47-51aac19036 -export IMPALA_TOOLCHAIN_BUILD_ID_X86_64=442-51aac19036 +export IMPALA_TOOLCHAIN_BUILD_ID_AARCH64=51-e06c7dcb96 +export IMPALA_TOOLCHAIN_BUILD_ID_X86_64=475-e06c7dcb96 export IMPALA_TOOLCHAIN_REPO=\ ${IMPALA_TOOLCHAIN_REPO:-https://github.com/cloudera/native-toolchain.git} export IMPALA_TOOLCHAIN_BRANCH=${IMPALA_TOOLCHAIN_BRANCH:-master} export IMPALA_TOOLCHAIN_COMMIT_HASH=\ -${IMPALA_TOOLCHAIN_COMMIT_HASH-51aac190369b543daddff2c0f29f54ded149a5f9} +${IMPALA_TOOLCHAIN_COMMIT_HASH-e06c7dcb9663a5df08a6bda025d82806762d019f} # Compare the build ref in build IDs by removing everything up-to-and-including the # first hyphen. if [ "${IMPALA_TOOLCHAIN_BUILD_ID_AARCH64#*-}" \ diff --git a/bin/jenkins/build-all-flag-combinations.sh b/bin/jenkins/build-all-flag-combinations.sh index 1143f706b..1b3bc28b8 100755 --- a/bin/jenkins/build-all-flag-combinations.sh +++ b/bin/jenkins/build-all-flag-combinations.sh @@ -127,7 +127,11 @@ else fi echo "Building with OPTIONS: $DESCRIPTION" - if ! time -p ./buildall.sh -skiptests -noclean ; then + # Since this is only testing the version replacements for Java, this uses -notests to + # skip building the backend tests. Because everything is already in ccache, building + # backend tests does many link invocations simultaneously and the system can run out of + # memory. + if ! time -p ./buildall.sh -notests -noclean ; then echo "Build failed: $DESCRIPTION" FAILED="${FAILED}:${DESCRIPTION}" fi
