On February 23, 2019 2:50:46 AM PST, Jeremie Courreges-Anglas <[email protected]>
wrote:
>On Sat, May 07 2016, Stefan Kempf <[email protected]> wrote:
>> Aaron Miller wrote:
>>> Hi All,
>>>
>>> I was experiencing ~8 minute linking times for a large C++
>application
>>> I have been working on when running -current on amd64. It turns out
>>> that the decade-old version of ld in the OpenBSD source tree has a
>bug
>>> that causes quadratic complexity for some linking operations when
>>> debug symbols are present. I found a patch from 2006 in a mailing
>list
>>> archive that fixes this; I adapted it to work with OpenBSD by
>editing
>>> files that are normally regenerated (I couldn't figure out how to do
>>> automatically regenerate them).
>>>
>>> Here is the original patch:
>>> https://sourceware.org/ml/binutils/2006-08/msg00334.html
>>>
>>> I have rebuilt the kernel and system with this ld and everything
>runs
>>> identically to the system linked with the unpatched ld.
>>>
>>> Please test this and let me know if this could get into the tree,
>and
>>> if not, what changes I need to make. Thanks!
>>
>> Interesting!
>>
>> About the autogenerated files: it looks you already made the
>additions
>> to bfd/linker.c, bfd/targets.c and bfd/libbfd.h that should end up
>> in bfd-in2.h and libbfd-in.h.
>>
>> To regenerate them, make sure you first did a "make obj" in the root
>of
>> your source tree. Then run "make depend" from gnu/usr.bin.
>Afterwards,
>> to to gnu/usr.bin/binutils-2.17/obj/bfd and run "make headers".
>> headers". That did the trick for me.
>
>A bit late to the party,
>
>I think this diff breaks ld.bfd(1) operations in some special cases.
>
>http://build-failures.rhaalovely.net//sparc64/2019-02-03/textproc/mupdf.log
>
>shows a double free on sparc64. A similar crash can be reproduced on
>amd64 using MALLOC_OPTIONS:
>
>--8<--
>russell /usr/ports/pobj/mupdf-1.14.0/mupdf-1.14.0-source$
>MALLOC_OPTIONS=S egdb --args ld.bfd -r -b binary -o
>build/release/resources/fonts/urw/NimbusMonoPS-Bold.cff.o
>resources/fonts/urw/NimbusMonoPS-Bold.cff
>GNU gdb (GDB) 7.12.1
>Copyright (C) 2017 Free Software Foundation, Inc.
>License GPLv3+: GNU GPL version 3 or later
><http://gnu.org/licenses/gpl.html>
>This is free software: you are free to change and redistribute it.
>There is NO WARRANTY, to the extent permitted by law. Type "show
>copying"
>and "show warranty" for details.
>This GDB was configured as "x86_64-unknown-openbsd6.4".
>Type "show configuration" for configuration details.
>For bug reporting instructions, please see:
><http://www.gnu.org/software/gdb/bugs/>.
>Find the GDB manual and other documentation resources online at:
><http://www.gnu.org/software/gdb/documentation/>.
>For help, type "help".
>Type "apropos word" to search for commands related to "word"...
>Reading symbols from ld.bfd...done.
>(gdb) r
>Starting program: /usr/bin/ld.bfd -r -b binary -o
>build/release/resources/fonts/urw/NimbusMonoPS-Bold.cff.o
>resources/fonts/urw/NimbusMonoPS-Bold.cff
>ld.bfd(20556) in free(): bogus pointer (double free?)
>0xdbdbdbdbdbdbdbdb
>
>Program received signal SIGABRT, Aborted.
>thrkill () at -:3
>3 -: No such file or directory.
>(gdb) bt
>#0 thrkill () at -:3
>#1 0x000007d40bd157fe in _libc_abort () at
>/usr/src/lib/libc/stdlib/abort.c:51
>#2 0x000007d40bca4909 in wrterror (d=0x7d3ab4b9be0, msg=0x7d40bc82ed6
>"bogus pointer (double free?) %p") at
>/usr/src/lib/libc/stdlib/malloc.c:297
>#3 0x000007d40bca7970 in findpool (p=0xdbdbdbdbdbdbdbdb,
>argpool=<optimized out>, foundpool=0x7f7ffffcb180,
>saved_function=0x7f7ffffcb178) at
>/usr/src/lib/libc/stdlib/malloc.c:1323
>#4 0x000007d40bca4af0 in ofree (argpool=0x7f7ffffcb1d8,
>p=0xdbdbdbdbdbdbdbdb, clear=0, check=0, argsz=0) at
>/usr/src/lib/libc/stdlib/malloc.c:1337
>#5 0x000007d40bca4a10 in free (ptr=0xdbdbdbdbdbdbdbdb) at
>/usr/src/lib/libc/stdlib/malloc.c:1451
>#6 0x000007d16b97be4a in bfd_elf_final_link (abfd=0x7d4006f0a00,
>info=0x7d16b9dc1f8 <link_info>) at
>/usr/src/gnu/usr.bin/binutils-2.17/bfd/elflink.c:8623
>#7 0x000007d16b8fe7f2 in ldwrite () at
>/usr/src/gnu/usr.bin/binutils-2.17/ld/ldwrite.c:557
>#8 0x000007d16b8fbde7 in main (argc=7, argv=0x7f7ffffcb778) at
>/usr/src/gnu/usr.bin/binutils-2.17/ld/ldmain.c:496
>(gdb) frame 6
>#6 0x000007d16b97be4a in bfd_elf_final_link (abfd=0x7d4006f0a00,
>info=0x7d16b9dc1f8 <link_info>) at
>/usr/src/gnu/usr.bin/binutils-2.17/bfd/elflink.c:8623
>8623 free (elf_tdata (sub)->symbuf);
>(gdb) l
>8618 /* Free symbol buffer if needed. */
>8619 if (!info->reduce_memory_overheads)
>8620 {
>8621 for (sub = info->input_bfds; sub != NULL; sub =
>sub->link_next)
>8622 if (elf_tdata (sub)->symbuf)
>8623 free (elf_tdata (sub)->symbuf);
>8624 }
>8625
>8626 /* Output any global symbols that got converted to local in a
>8627 version script or due to symbol visibility. We do this in
>a
>(gdb)
>-->8--
>
>This is not specific to mupdf, any source file will do. I have no idea
>yet where elf_tdata(sub) gets allocated in this case.
>
>Using --reduce-memory-overheads appears to work around the problem.
Hi Jeremie,
That is concerning. I'm on my phone and haven't had a chance to investigate,
but from the code in the gdb output above, it looks like the author of the diff
forgot to set the pointer to NULL after freeing. For example:
if (elf_tdata (sub)->symbuf) {
free (elf_tdata (sub)->symbuf);
elf_tdata (sub)->symbuf = NULL;
}
This is not tested at all. I hope this works!
--Aaron
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
