[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 H.J. Lu changed: What|Removed |Added Target Milestone|--- |2.43 Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #12 from H.J. Lu --- Fixed for 2.43. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 --- Comment #11 from Sourceware Commits --- The master branch has been updated by H.J. Lu : https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=3428c771aef0cebe229ac0f02a654d2ef78f2427 commit 3428c771aef0cebe229ac0f02a654d2ef78f2427 Author: H.J. Lu Date: Fri Mar 8 16:21:42 2024 -0800 elf: Always keep symbol table and relocation info for eh_frame When --no-keep-memory is used, the symbol table and relocation info for eh_frame are freed after they are retrieved for each text section in the input object. If an input object has many text sections, the same data is retrieved and freed many times which can take a very long time. Update _bfd_elf_gc_mark to keep the symbol table and relocation info for eh_frame to avoid it. Data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: before after improvement user86.31 86.44 -0.2% system 8.778.631.6% total 95.58 96.81 -1.3% maximum set(GB) 13.113.10% page faults 3024752 3028699 -1.3% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user5.495.46-0.5% system 0.730.730% total 6.266.250.3% maximum set(MB) 964 964 0% page faults 235173 235796 -0.3% The memory usage impact is minimum and the link time of the Rust binary in https://sourceware.org/bugzilla/show_bug.cgi?id=31466 is reduced from 500+ seconds to 1.44 seconds, a 300x speedup. PR ld/31466 * elflink.c (init_reloc_cookie): Add a bool argument, keep_memory, for keeping memory. Always keep memory if keep_memory is true. (init_reloc_cookie_rels): Likewise (init_reloc_cookie_for_section): Add a bool argument for keeping memory and pass it to init_reloc_cookie and init_reloc_cookie_rels. (_bfd_elf_gc_mark_reloc): Pass false to _bfd_elf_gc_mark. (_bfd_elf_gc_mark): Pass true to init_reloc_cookie_for_section for the eh_frame section. Pass false to init_reloc_cookie_for_section for other sections. (_bfd_elf_gc_mark_extra_sections): Add Add a bool argument for keeping memory and pass it to _bfd_elf_gc_mark. (bfd_elf_parse_eh_frame_entries): Pass false to init_reloc_cookie and init_reloc_cookie_rels. (bfd_elf_gc_sections): Pass false to init_reloc_cookie_for_section and _bfd_elf_gc_mark. (bfd_elf_discard_info): Pass false to init_reloc_cookie_for_section and init_reloc_cookie. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 --- Comment #10 from H.J. Lu --- (In reply to Alan Modra from comment #9) > Perhaps it would be better to just drop the mmap change in > _bfd_elf_link_keep_memory. I did wonder when reviewing that patch whether > it was a good idea to discard things just based on whether mmap was enabled. I will add --keep-memory to always cache the symbol and relocation tables. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 --- Comment #9 from Alan Modra --- Perhaps it would be better to just drop the mmap change in _bfd_elf_link_keep_memory. I did wonder when reviewing that patch whether it was a good idea to discard things just based on whether mmap was enabled. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 --- Comment #8 from H.J. Lu --- Created attachment 15395 --> https://sourceware.org/bugzilla/attachment.cgi?id=15395=edit This patch is on top of the mmap patches -- You are receiving this mail because: You are on the CC list for the bug.
[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 --- Comment #7 from Sam James --- That's looking good, thanks! I'll run it overnight on top of mmap (needed trivial rebasing) with the package builds. If needed, I can do a separate large build with -Wl,--no-keep-memory and just this patch tomorrow. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 --- Comment #6 from H.J. Lu --- Created attachment 15394 --> https://sourceware.org/bugzilla/attachment.cgi?id=15394=edit Try this -- You are receiving this mail because: You are on the CC list for the bug.
[Bug ld/31466] It takes a long time to build Rust program with --no-keep-memory
https://sourceware.org/bugzilla/show_bug.cgi?id=31466 H.J. Lu changed: What|Removed |Added Summary|Hang when building Rust |It takes a long time to |program with|build Rust program with |--no-keep-memory|--no-keep-memory --- Comment #5 from H.J. Lu --- It doesn't hang. It just takes a LONG time: $ /bin/time ../ld.master --no-keep-memory `cat doit` 371.06user 180.83system 9:13.36elapsed 99%CPU (0avgtext+0avgdata 1185900maxresident)k 0inputs+50408outputs (0major+71941278minor)pagefaults 0swaps -- You are receiving this mail because: You are on the CC list for the bug.
