[
https://issues.apache.org/jira/browse/IMPALA-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788556#comment-17788556
]
Michael Smith edited comment on IMPALA-12563 at 11/21/23 9:25 PM:
------------------------------------------------------------------
I got UBSAN builds working. All tests pass except FE tests, which fail with
{code}
[ERROR] Run 1: JniFrontendTest.setup:49 » UnsatisfiedLink
/home/msmith2.linux/impala/be/build/debug/service/libfesupport.so:
/home/msmith2.linux/impala/be/build/debug/service/libfesupport.so: cannot
allocate memory in static TLS block
{code}
I compared libfesupport.so and all the libraries it depends on between aarch64
and x86_64 platforms; the only difference in TLS allocation is that libjvm.so
allocates 40 bytes on aarch64 (and none on x86_64).
{code}
$ readelf -Wl ./be/build/debug/service/libfesupport.so | grep TLS
TLS 0x1322fd40 0x0000000013230d40 0x0000000013230d40 0x000080
0x000140 R 0x40
$ for p in $(ldd be/build/debug/service/libfesupport.so | grep '=>' | cut -d' '
-f3); do readelf -Wl $p | grep TLS && echo $p; done
TLS 0x7c3900 0x00000000007c4900 0x00000000007c4900 0x000080
0x000100 R 0x40
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/release/lib/libkudu_client.so.0
TLS 0x012cc0 0x0000000000212cc0 0x0000000000212cc0 0x000000
0x000008 R 0x8
/home/michael/Impala/toolchain/cdp_components-45689292/hadoop-3.1.1.7.2.18.0-369/lib/native/libhdfs.so.0.0.0
TLS 0x1c3978 0x00000000001c4978 0x00000000001c4978 0x000000
0x000020 R 0x8
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/calloncehack-1.0.0/lib/libstdc++.so.6
TLS 0x2148f0 0x00000000002158f0 0x00000000002158f0 0x000010
0x000090 R 0x8
/lib/x86_64-linux-gnu/libc.so.6
TLS 0x003ca8 0x0000000000004ca8 0x0000000000004ca8 0x000000
0x000019 R 0x1
/lib/x86_64-linux-gnu/libcom_err.so.2
{code}
vs
{code}
$ readelf -Wl ./be/build/debug/service/libfesupport.so | grep TLS
TLS 0x117086c0 0x00000000117186c0 0x00000000117186c0 0x000080
0x000140 R 0x40
$ for p in $(ldd be/build/debug/service/libfesupport.so | grep '=>' | cut -d' '
-f3); do readelf -Wl $p | grep TLS && echo $p; done
TLS 0xc5cd58 0x0000000000c6cd58 0x0000000000c6cd58 0x000000
0x000028 R 0x8
/usr/lib/jvm/java-8-openjdk-arm64/jre/lib/aarch64/server/libjvm.so
TLS 0x715d00 0x0000000000725d00 0x0000000000725d00 0x000080
0x000100 R 0x40
/home/msmith2.linux/impala/toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/release/lib/libkudu_client.so.0
TLS 0x00fc90 0x000000000001fc90 0x000000000001fc90 0x000000
0x000008 R 0x8
/home/msmith2.linux/impala/toolchain/cdp_components-45689292/hadoop-3.1.1.7.2.18.0-369/lib/native/libhdfs.so.0.0.0
TLS 0x1bf7e0 0x00000000001cf7e0 0x00000000001cf7e0 0x000000
0x000020 R 0x8
/home/msmith2.linux/impala/toolchain/toolchain-packages-gcc10.4.0/calloncehack-1.0.0/lib/libstdc++.so.6
TLS 0x188da8 0x0000000000198da8 0x0000000000198da8 0x000010
0x000090 R 0x8
/lib/aarch64-linux-gnu/libc.so.6
TLS 0x002c90 0x0000000000012c90 0x0000000000012c90 0x000000
0x000019 R 0x8
/lib/aarch64-linux-gnu/libcom_err.so.2
{code}
https://bugzilla.redhat.com/show_bug.cgi?id=1722181 has some discussion around
aarch64, noting
{quote}
The GNU TLS2 model which I'm afraid aarch64 uses unfortunately eats from the
same TLS preallocated pool as libraries that require static TLS like libgomp,
where it is performance critical to have it as static TLS.
{quote}
However I don't have an explanation why this only shows up with UBSAN builds.
Dynamic library TLS reservations are the same with UBSAN and non-UBSAN builds.
For now I'm disabling FE_TEST for UBSAN builds on aarch64.
was (Author: JIRAUSER288956):
I got UBSAN builds working. All tests pass except FE tests, which fail with
{code}
[ERROR] Run 1: JniFrontendTest.setup:49 » UnsatisfiedLink
/home/msmith2.linux/impala/be/build/debug/service/libfesupport.so:
/home/msmith2.linux/impala/be/build/debug/service/libfesupport.so: cannot
allocate memory in static TLS block
{code}
I compared libfesupport.so and all the libraries it depends on between aarch64
and x86_64 platforms; the only difference in TLS allocation is that libjvm.so
allocates 40 bytes on aarch64 (and none on x86_64).
{code}
$ readelf -Wl ./be/build/debug/service/libfesupport.so | grep TLS ✘
130
TLS 0x1322fd40 0x0000000013230d40 0x0000000013230d40 0x000080
0x000140 R 0x40
$ for p in $(ldd be/build/debug/service/libfesupport.so | grep '=>' | cut -d' '
-f3); do echo $p && readelf -Wl $p | grep TLS; done
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/calloncehack-1.0.0/lib/libgcc_s.so.1
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/calloncehack-1.0.0/lib/libcalloncehack.so
/lib/x86_64-linux-gnu/libsasl2.so.2
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/release/lib/libkudu_client.so.0
TLS 0x7c3900 0x00000000007c4900 0x00000000007c4900 0x000080
0x000100 R 0x40
/home/michael/Impala/toolchain/cdp_components-45689292/hadoop-3.1.1.7.2.18.0-369/lib/native/libhdfs.so.0.0.0
TLS 0x012cc0 0x0000000000212cc0 0x0000000000212cc0 0x000000
0x000008 R 0x8
/lib/x86_64-linux-gnu/libkrb5.so.3
/lib/x86_64-linux-gnu/libgssapi_krb5.so.2
/lib/x86_64-linux-gnu/libssl.so.3
/lib/x86_64-linux-gnu/libcrypto.so.3
/home/michael/Impala/toolchain/toolchain-packages-gcc10.4.0/calloncehack-1.0.0/lib/libstdc++.so.6
TLS 0x1c3978 0x00000000001c4978 0x00000000001c4978 0x000000
0x000020 R 0x8
/lib/x86_64-linux-gnu/libm.so.6
/lib/x86_64-linux-gnu/libc.so.6
TLS 0x2148f0 0x00000000002158f0 0x00000000002158f0 0x000010
0x000090 R 0x8
/lib/x86_64-linux-gnu/libdl.so.2
/lib/x86_64-linux-gnu/libpthread.so.0
/lib/x86_64-linux-gnu/libk5crypto.so.3
/lib/x86_64-linux-gnu/libcom_err.so.2
TLS 0x003ca8 0x0000000000004ca8 0x0000000000004ca8 0x000000
0x000019 R 0x1
/lib/x86_64-linux-gnu/libkrb5support.so.0
/lib/x86_64-linux-gnu/libkeyutils.so.1
/lib/x86_64-linux-gnu/libresolv.so.2
{code}
vs
{code}
$ readelf -Wl ./be/build/debug/service/libfesupport.so | grep TLS
TLS 0x117086c0 0x00000000117186c0 0x00000000117186c0 0x000080
0x000140 R 0x40
$ for p in $(ldd be/build/debug/service/libfesupport.so | grep '=>' | cut -d' '
-f3); do echo $p && readelf -Wl $p | grep TLS; done
/usr/lib/jvm/java-8-openjdk-arm64/jre/lib/aarch64/server/libjvm.so
TLS 0xc5cd58 0x0000000000c6cd58 0x0000000000c6cd58 0x000000
0x000028 R 0x8
/home/msmith2.linux/impala/toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/release/lib/libkudu_client.so.0
TLS 0x715d00 0x0000000000725d00 0x0000000000725d00 0x000080
0x000100 R 0x40
/home/msmith2.linux/impala/toolchain/cdp_components-45689292/hadoop-3.1.1.7.2.18.0-369/lib/native/libhdfs.so.0.0.0
TLS 0x00fc90 0x000000000001fc90 0x000000000001fc90 0x000000
0x000008 R 0x8
/lib/aarch64-linux-gnu/libkrb5.so.3
/lib/aarch64-linux-gnu/libgssapi_krb5.so.2
/lib/aarch64-linux-gnu/libssl.so.3
/lib/aarch64-linux-gnu/libcrypto.so.3
/home/msmith2.linux/impala/toolchain/toolchain-packages-gcc10.4.0/calloncehack-1.0.0/lib/libstdc++.so.6
TLS 0x1bf7e0 0x00000000001cf7e0 0x00000000001cf7e0 0x000000
0x000020 R 0x8
/lib/aarch64-linux-gnu/libm.so.6
/lib/aarch64-linux-gnu/libc.so.6
TLS 0x188da8 0x0000000000198da8 0x0000000000198da8 0x000010
0x000090 R 0x8
/lib/aarch64-linux-gnu/libpthread.so.0
/lib/aarch64-linux-gnu/libk5crypto.so.3
/lib/aarch64-linux-gnu/libcom_err.so.2
TLS 0x002c90 0x0000000000012c90 0x0000000000012c90 0x000000
0x000019 R 0x8
/lib/aarch64-linux-gnu/libkrb5support.so.0
/lib/aarch64-linux-gnu/libkeyutils.so.1
/lib/aarch64-linux-gnu/libresolv.so.2
{code}
https://bugzilla.redhat.com/show_bug.cgi?id=1722181 has some discussion around
aarch64, noting
{quote}
The GNU TLS2 model which I'm afraid aarch64 uses unfortunately eats from the
same TLS preallocated pool as libraries that require static TLS like libgomp,
where it is performance critical to have it as static TLS.
{quote}
However I don't have an explanation why this only shows up with UBSAN builds.
Dynamic library TLS reservations are the same with UBSAN and non-UBSAN builds.
For now I'm disabling FE_TEST for UBSAN builds on aarch64.
> ARM UBSAN build fails with undefined reference to aarch symbols
> ---------------------------------------------------------------
>
> Key: IMPALA-12563
> URL: https://issues.apache.org/jira/browse/IMPALA-12563
> Project: IMPALA
> Issue Type: Task
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Major
>
> Building with UBSAN on ARM fails with
> {code}
> /data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/glog-0.3.5-p3/lib/libglog.a(libglog_la-raw_logging.o):raw_logging.cc:function
> google::RawLog__(int, char const*, int, char const*, ...): error: undefined
> reference to '__aarch64_cas1_sync'
> /data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/protobuf-3.14.0-clangcompat-p2/lib/libprotobuf.a(common.o):common.cc:function
> google::protobuf::LogSilencer::LogSilencer(): error: undefined reference to
> '__aarch64_ldadd4_acq_rel'
> /data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/protobuf-3.14.0-clangcompat-p2/lib/libprotobuf.a(common.o):common.cc:function
> google::protobuf::LogSilencer::~LogSilencer(): error: undefined reference to
> '__aarch64_ldadd4_acq_rel'
> /data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/protobuf-3.14.0-clangcompat-p2/lib/libprotobuf.a(arena.o):arena.cc:function
> google::protobuf::internal::ArenaImpl::Init(bool): error: undefined
> reference to '__aarch64_ldadd8_relax'
> /data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/protobuf-3.14.0-clangcompat-p2/lib/libprotobuf.a(arena.o):arena.cc:function
> google::protobuf::internal::ArenaImpl::NewBuffer(unsigned long, unsigned
> long): error: undefined reference to '__aarch64_ldadd8_relax'
> /data/jenkins/workspace/impala-private-basic-parameterized/Impala-Toolchain/toolchain-packages-gcc10.4.0/protobuf-3.14.0-clangcompat-p2/lib/libprotobuf.a(arena.o):arena.cc:function
> google::protobuf::internal::ArenaImpl::GetSerialArenaFallback(void*): error:
> undefined reference to '__aarch64_cas8_rel'
> clang-5.0: error: linker command failed with exit code 1 (use -v to see
> invocation)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
