[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. IMPALA-12334: Add debug symbols to LLVM assert build Builds the LLVM debug build (-asserts) with -g1 so we get reasonable stack traces in LLVM while debugging Impala. Excludes building some parts of LLVM, similar to what Kudu does in https://github.com/apache/kudu/blob/master/thirdparty/build-definitions.sh. Strips binaries we don't link against at runtime as the goal is to debug Impala's use of LLVM, not LLVM itself. These changes shrink the non-assert LLVM build by ~100 MB, and keep the assert build around the same size. Using RelWithDebInfo increases the size from ~350MB to 1GB. Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Reviewed-on: http://gerrit.cloudera.org:8080/20311 Reviewed-by: Joe McDonnell Tested-by: Michael Smith --- M source/llvm/build-source-tarball.sh 1 file changed, 90 insertions(+), 1 deletion(-) Approvals: Joe McDonnell: Looks good to me, approved Michael Smith: Verified -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 14 Aug 2023 20:42:12 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Patch Set 5: Code-Review+2 This looks good -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 14 Aug 2023 19:46:50 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Michael Smith has posted comments on this change. (
http://gerrit.cloudera.org:8080/20311 )
Change subject: IMPALA-12334: Add debug symbols to LLVM assert build
..
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20311/4/source/llvm/build-source-tarball.sh
File source/llvm/build-source-tarball.sh:
http://gerrit.cloudera.org:8080/#/c/20311/4/source/llvm/build-source-tarball.sh@184
PS4, Line 184: function strip_if_possible() {
: filename=$1
: if [[ "$(file -bi $filename)" =
application/x-@(executable|sharedlib|archive)* ]]
: then
: strip -gx "$filename"
: fi
: }
:
> Nit: This works, but it's a bit hard for me to follow. Could we do somethin
Done
--
To view, visit http://gerrit.cloudera.org:8080/20311
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8
Gerrit-Change-Number: 20311
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Yida Wu
Gerrit-Comment-Date: Thu, 10 Aug 2023 16:50:13 +
Gerrit-HasComments: Yes
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Hello Yida Wu, Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20311 to look at the new patch set (#5). Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. IMPALA-12334: Add debug symbols to LLVM assert build Builds the LLVM debug build (-asserts) with -g1 so we get reasonable stack traces in LLVM while debugging Impala. Excludes building some parts of LLVM, similar to what Kudu does in https://github.com/apache/kudu/blob/master/thirdparty/build-definitions.sh. Strips binaries we don't link against at runtime as the goal is to debug Impala's use of LLVM, not LLVM itself. These changes shrink the non-assert LLVM build by ~100 MB, and keep the assert build around the same size. Using RelWithDebInfo increases the size from ~350MB to 1GB. Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 --- M source/llvm/build-source-tarball.sh 1 file changed, 90 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/11/20311/5 -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/20311 )
Change subject: IMPALA-12334: Add debug symbols to LLVM assert build
..
Patch Set 4:
(1 comment)
Thanks for looking at this, having these symbols will be useful
http://gerrit.cloudera.org:8080/#/c/20311/4/source/llvm/build-source-tarball.sh
File source/llvm/build-source-tarball.sh:
http://gerrit.cloudera.org:8080/#/c/20311/4/source/llvm/build-source-tarball.sh@184
PS4, Line 184: local strip='for pathname do
: if [[ "$(file -bi "$pathname")" =
application/x-@(executable|sharedlib|archive)* ]]
: then
: strip -gx "$pathname"
: fi; done'
: find ${LOCAL_INSTALL}/bin -type f -exec bash -c "$strip" bash
{} +
: find ${LOCAL_INSTALL}/lib -type f \( -iname "libclang*" -o
-name "libLTO*" \) \
: -exec bash -c "$strip" bash {} +
Nit: This works, but it's a bit hard for me to follow. Could we do something
like this (probably some bugs in here):
function strip_if_possible() {
filename = $1
if [[ "$(file -bi $filename)" =
application/x-@(executable|sharedlib|archive)* ]]; then
strip -gx "$filename"
fi
}
for binary in $(find ${LOCAL_INSTALL}/bin -type f); do
strip_if_possible $binary
done
for lib in $(find ${LOCAL_INSTALL}/lib -type f \( -iname "libclang*" -o -name
"libLTO*" \)); do
strip_if_possible $lib
done
--
To view, visit http://gerrit.cloudera.org:8080/20311
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8
Gerrit-Change-Number: 20311
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Yida Wu
Gerrit-Comment-Date: Thu, 10 Aug 2023 00:08:53 +
Gerrit-HasComments: Yes
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Hello Yida Wu, Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20311 to look at the new patch set (#4). Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. IMPALA-12334: Add debug symbols to LLVM assert build Builds the LLVM debug build (-asserts) with -g1 so we get reasonable stack traces in LLVM while debugging Impala. Excludes building some parts of LLVM, similar to what Kudu does in https://github.com/apache/kudu/blob/master/thirdparty/build-definitions.sh. Strips binaries we don't link against at runtime as the goal is to debug Impala's use of LLVM, not LLVM itself. These changes shrink the non-assert LLVM build by ~100 MB, and keep the assert build around the same size. Using RelWithDebInfo increases the size from ~350MB to 1GB. Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 --- M source/llvm/build-source-tarball.sh 1 file changed, 83 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/11/20311/4 -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Patch Set 3: > Patch Set 3: > > What happens to the size of the LLVM tarball? I remember trying -g at some > point and the disk space usage was large. Yes, that's what I just found. Tarball is 11GB. That's unfortunate. I'm going to see if I can limit it to libraries and not the executables. -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 07 Aug 2023 17:17:45 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Michael Smith has removed a vote on this change. Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Removed Code-Review+1 by Michael Smith -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Patch Set 3: What happens to the size of the LLVM tarball? I remember trying -g at some point and the disk space usage was large. -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Mon, 07 Aug 2023 17:01:03 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Thu, 03 Aug 2023 18:31:38 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Hello Yida Wu, Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20311 to look at the new patch set (#3). Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. IMPALA-12334: Add debug symbols to LLVM assert build Builds the LLVM debug build (-asserts) with RelWithDebInfo so we can step into LLVM code while debugging Impala. Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 --- M source/llvm/build-source-tarball.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/11/20311/3 -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Patch Set 2: Paste a link of related doc: https://llvm.org/docs/CMake.html#cmake-build-type -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Thu, 03 Aug 2023 18:05:50 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20311 ) Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu Gerrit-Comment-Date: Thu, 03 Aug 2023 18:04:42 + Gerrit-HasComments: No
[native-toolchain-CR] IMPALA-12334: Add debug symbols to LLVM assert build
Hello Yida Wu, Joe McDonnell, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20311 to look at the new patch set (#2). Change subject: IMPALA-12334: Add debug symbols to LLVM assert build .. IMPALA-12334: Add debug symbols to LLVM assert build Builds the LLVM debug build (-assert) with RelWithDebInfo so we can step into LLVM code while debugging Impala. Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 --- M source/llvm/build-source-tarball.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/11/20311/2 -- To view, visit http://gerrit.cloudera.org:8080/20311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: native-toolchain Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9c58bfe2233dcec39fcad92fc52ce372165229d8 Gerrit-Change-Number: 20311 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Yida Wu
