[
https://issues.apache.org/jira/browse/IMPALA-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith closed IMPALA-14527.
----------------------------------
Resolution: Won't Do
mold 2.40.4 results in a binary that fails our SASL/GSSAPI tests. We see
backend testing fail with
{code}
/home/ubuntu/Impala/be/src/rpc/thrift-server-test.cc:221
Value of: status_.ok()
Actual: false
Expected: true
Error: Couldn't open transport for localhost:60081 (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more information
(Server krbtgt/[email protected] not found in Kerberos database))
{code}
and similar errors in logs during frontend customcluster tests.
> Switch to mold as the default linker
> ------------------------------------
>
> Key: IMPALA-14527
> URL: https://issues.apache.org/jira/browse/IMPALA-14527
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Major
>
> The mold linker better utilizes concurrency to speed up linking a single
> binary. This has two benefits
> 1. Development when relinking e.g. impalad will be significantly faster (
> 2. During a full build that links many test binaries, we can use
> MOLD_LINKER=1 to link a single binary at a time - using significantly less
> memory - while still fully utilizing all CPU cores.
> Full builds are also a bit faster. After priming ccache with a full build;
> with mold rebuilds take ~2m33s and memory use remains under 10GiB (24 cores)
> {code}
> $ env MOLD_JOBS=1 IMPALA_LINKER=mold time ./buildall.sh -skiptests
> ...
> 313.23user 67.68system 2:33.65elapsed 247%CPU (0avgtext+0avgdata
> 5664100maxresident)k
> 16inputs+34793584outputs (216major+12078088minor)pagefaults 0swaps
> {code}
> With gold, it takes ~3m15s and maxes out my memory (64GiB) requiring
> additional swap
> {code}
> $ env IMPALA_LINKER=gold time ./buildall.sh -skiptests
> ...
> 261.36user 71.43system 3:15.41elapsed 170%CPU (0avgtext+0avgdata
> 3655132maxresident)k
> 7485904inputs+34794136outputs (88605major+11836509minor)pagefaults 0swaps
> {code}
> Performance of the resulting binaries looks essentially the same:
> https://jenkins.impala.io/job/perf-AB-test-ub2004/336/artifact/Impala/perf_results/latest/performance_result.txt.
> Let's switch to using IMPALA_LINKER=mold by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
