Michael Smith created IMPALA-14527:
--------------------------------------
Summary: Switch to mold as the default linker
Key: IMPALA-14527
URL: https://issues.apache.org/jira/browse/IMPALA-14527
Project: IMPALA
Issue Type: Task
Components: Infrastructure
Reporter: Michael Smith
The mold linker better utilizes concurrency to speed up linking a single
binary. This has two benefits
1. Development when relinking e.g. impalad will be significantly faster (
2. During a full build that links many test binaries, we can use MOLD_LINKER=1
to link a single binary at a time - using significantly less memory - while
still fully utilizing all CPU cores.
Full builds are also a bit faster. After priming ccache with a full build; with
mold rebuilds take ~2m33s and memory use remains under 10GiB (24 cores)
{code}
$ env MOLD_JOBS=1 IMPALA_LINKER=mold time ./buildall.sh -skiptests
...
313.23user 67.68system 2:33.65elapsed 247%CPU (0avgtext+0avgdata
5664100maxresident)k
16inputs+34793584outputs (216major+12078088minor)pagefaults 0swaps
{code}
With gold, it takes ~3m15s and maxes out my memory (64GiB) requiring additional
swap
{code}
$ env IMPALA_LINKER=gold time ./buildall.sh -skiptests
...
261.36user 71.43system 3:15.41elapsed 170%CPU (0avgtext+0avgdata
3655132maxresident)k
7485904inputs+34794136outputs (88605major+11836509minor)pagefaults 0swaps
{code}
Performance of the resulting binaries looks essentially the same:
https://jenkins.impala.io/job/perf-AB-test-ub2004/336/artifact/Impala/perf_results/latest/performance_result.txt.
Let's switch to using IMPALA_LINKER=mold by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
