[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #16 from Martin Nowak --- Regarding the dlopen/dlclose in handleForName, the semantics of RTLD_NOLOAD are so that it bumps the reference count if the library had been previously loaded. The sections module uses the handle as identifier to distinguish different DSOs and store metadata. Due to the self-registration mechanism of D DSOs (see _d_dso_registry) any D library remains loaded and (presumably) keeps the same handle if it is needed by the executable and loaded during startup. If a shared library is loaded dynamically, then the dlopen/dlclose actions will surround the calls to _d_dso_registry by the library, hence the handle should stay valid while until the DSO is unregistered. This is a lot of low-level plumbing (without going into private runtime linker structs) and semantics have been modeled after Linux and FreeBSD. If they differ on Solaris the code might indeed need some fixup. The RTLD_NOLOAD behavior of only inc-refing already loaded libs isn't well documented but seems like the only sensible one.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 ibuclaw at gcc dot gnu.org changed: What|Removed |Added Status|UNCONFIRMED |RESOLVED CC||ibuclaw at gcc dot gnu.org Resolution|--- |FIXED --- Comment #15 from ibuclaw at gcc dot gnu.org --- See r270345 and r270347.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #14 from Rainer Orth --- Author: ro Date: Sun Apr 14 09:30:42 2019 New Revision: 270347 URL: https://gcc.gnu.org/viewcvs?rev=270347&root=gcc&view=rev Log: Work around lack of dlpi_tls_modid before Solaris 11.5 2019-04-14 Rainer Orth Iain Buclaw PR d/88150 * m4/druntime/os.m4 (DRUNTIME_OS_DLPI_TLS_MODID): New macro. * configure.ac: Use it. Call AC_USE_SYSTEM_EXTENSIONS. * configure: Regenerate. * Makefile.in, libdruntime/Makefile.in, src/Makefile.in, testsuite/Makefile.in: Regenerate. * libdruntime/gcc/config.d.in (OS_Have_Dlpi_Tls_Modid): Define. * libdruntime/gcc/sections/elf_shared.d: Import gcc.config. (scanSegments) [OS_Have_Dlpi_Tls_Modid]: Use dlpi_tls_modid. [Solaris]: Use dlinfo(RTLD_DI_LINKMAP) to get rt_tlsmodid. Otherwise clear pdso._tlsMod, pdso._tlsSize. (getTLSRange) [Solaris && !OS_Have_Dlpi_Tls_Modid]: Readjust mod. Modified: trunk/libphobos/ChangeLog trunk/libphobos/Makefile.in trunk/libphobos/configure (contents, props changed) trunk/libphobos/configure.ac trunk/libphobos/libdruntime/Makefile.in trunk/libphobos/libdruntime/gcc/config.d.in trunk/libphobos/libdruntime/gcc/sections/elf_shared.d trunk/libphobos/m4/druntime/os.m4 trunk/libphobos/src/Makefile.in trunk/libphobos/testsuite/Makefile.in
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #13 from Rainer Orth --- Author: ro Date: Sun Apr 14 09:18:42 2019 New Revision: 270345 URL: https://gcc.gnu.org/viewcvs?rev=270345&root=gcc&view=rev Log: Use gcc/sections/elf_shared.d on Solaris 11.5 (PR d/88150) PR d/88150 * libdruntime/gcc/sections/elf_shared.d [Solaris] (SharedELF): Set to true. Import core.sys.solaris.dlfcn, core.sys.solaris.link, core.sys.solaris.sys.elf, core.sys.solaris.sys.link. (dummy_ref): Declare. (initSections): Initialize dummy_ref. (getDependencies): Set strtab. (handleForName): Don't dlclose handle. (findDSOInfoForAddr): Set IterateManually. (getprogname): Declare. (progname): Use it. * libdruntime/gcc/sections/package.d [Solaris]: Import gcc.sections.elf_shared instead of gcc.sections.solaris. * libdruntime/gcc/sections/solaris.d: Remove. * libdruntime/Makefile.am (DRUNTIME_DSOURCES): Remove gcc/sections/solaris.d. Removed: trunk/libphobos/libdruntime/gcc/sections/solaris.d Modified: trunk/libphobos/ChangeLog trunk/libphobos/libdruntime/Makefile.am trunk/libphobos/libdruntime/Makefile.in trunk/libphobos/libdruntime/gcc/sections/elf_shared.d trunk/libphobos/libdruntime/gcc/sections/package.d
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #12 from ro at CeBiTec dot Uni-Bielefeld.DE --- I've now reworked my non-dlpi_tls_modid patch to include this after Solaris 11.[345]/x86 testing gave excellent and pretty much identical test results: https://gcc.gnu.org/ml/gcc-patches/2019-04/msg00354.html
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #11 from ro at CeBiTec dot Uni-Bielefeld.DE --- > --- Comment #10 from Iain Buclaw --- > I've got a (horrible?) hack for getting tls_modid from Solaris. Cool, and not really horrible ;-) It's already this way in Solaris 9 (I happen to have access to those sources and just checked), so we can assume this is stable. > Looking at dlinfo(RTLD_DI_LINKMAP): > > https://github.com/illumos/illumos-gate/blob/4e0c5eff9af325c80994e9527b7cb8b3a1ffd1d4/usr/src/cmd/sgs/rtld/common/dlfcns.c#L1927-L1934 I missed that because I initially looked only at public headers for tlsmodid, and Rt_map is private to ld.so.1. This is intentionally missing from dlinfo(3C), too. I only found it in 's rd_loadobj_t, but it seemed hard to apply librtld_db to the current process instead of one being debugged... > Interestingly, Solaris sets the first tlsmodid to zero. > > https://github.com/illumos/illumos-gate/blob/8a06b3d6467c15646e663c05086378f16288af85/usr/src/cmd/sgs/rtld/common/tls.c#L52-L60 > > This is in stark contrast to what elf_shared expects, given that there's an > assert that it is always non-zero if _tlsSize is set. Right: 0 had been used for the tlsmodid of the main executable until dlpi_tls_modid was added to dl_phdr_info for Solaris 11.5 (still in beta) when this would-be incompatibility with the Linux and BSD versions was noticed. It was only changed to 1 once it could be determined that it was safe to do so.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #10 from Iain Buclaw --- I've got a (horrible?) hack for getting tls_modid from Solaris. Looking at dlinfo(RTLD_DI_LINKMAP): https://github.com/illumos/illumos-gate/blob/4e0c5eff9af325c80994e9527b7cb8b3a1ffd1d4/usr/src/cmd/sgs/rtld/common/dlfcns.c#L1927-L1934 The lmp variable returned is originally an Rt_map*, layout is: --- struct Rt_map { Link_map rt_public; const char* rt_pathname; c_ulong rt_padstart; c_ulong rt_padimlen; c_ulong rt_msize; uint rt_flags; uint rt_flags1; c_ulong rt_tlsmodid; } --- Let's do a little test... --- case PT_TLS: // TLS segment assert(!pdso._tlsSize); // is unique per DSO static if (OS_Have_Dlpi_Tls_Modid) { pdso._tlsMod = info.dlpi_tls_modid; pdso._tlsSize = phdr.p_memsz; } else version (Solaris) { Rt_map* map; version (Shared) dlinfo(handleForName(info.dlpi_name), RTLD_DI_LINKMAP, &map); else dlinfo(RTLD_SELF, RTLD_DI_LINKMAP, &map); pdso._tlsMod = map.rt_tlsmodid; pdso._tlsSize = phdr.p_memsz; } else { pdso._tlsMod = 0; pdso._tlsSize = 0; } break; --- Inspecting this in gdb --- (gdb) p map $1 = (struct Rt_map *) 0xfedb001 (gdb) p *map $2 = {rt_public = {l_addr = 4275896320, l_name = 0xfc0f02e4 "/mnt/build/i386-pc-solaris2.11/./libphobos/libdruntime/.libs/libgdruntime.so.76", l_ld = 0xfedd00d4, l_next = 0xfedb0678, l_prev = 0xfef206c8, l_refname = 0x0}, rt_pathname = 0xfc0f0334 "/mnt/build/i386-pc-solaris2.11/libphobos/libdruntime/.libs/libgdruntime.so.76.0.3", rt_padstart = 4275896320, rt_padimlen = 1272912, rt_msize = 1272912, rt_flags = 272761348, rt_flags1 = 1155, rt_tlsmodid = 2} --- And there it is, a valid rt_tlsmodid. Interestingly, Solaris sets the first tlsmodid to zero. https://github.com/illumos/illumos-gate/blob/8a06b3d6467c15646e663c05086378f16288af85/usr/src/cmd/sgs/rtld/common/tls.c#L52-L60 This is in stark contrast to what elf_shared expects, given that there's an assert that it is always non-zero if _tlsSize is set.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #9 from ro at CeBiTec dot Uni-Bielefeld.DE --- > --- Comment #8 from Johannes Pfau --- > Regarding the _d_dso_registry issue: Yes, as far as I can see it is a bug that > handleForName dlcloses the handle here. I think what happened is this: > > handleForName is used in one place with the comment > // get handle without loading the library > so it is supposed to unload the library there. > But it is also called from handleForAddr which is used to get the DSO handle > to > be stored using setDSOForHandle. I think here, it's not really valid to store > the closed handle. If the dlclose is indeed invalid, my workaround of disabling it explains 3 of the 4 remaining libphobos.shared failures. If I disable the dlclose unconditionally and rerun the libphobos.shared tests on Linux/x86_64, I get those failures there, too, both 64 and 32-bit: FAIL: libphobos.shared/load.d -shared-libphobos -ldl execution test FAIL: libphobos.shared/load_13414.d -shared-libphobos -ldl execution test FAIL: libphobos.shared/finalize.d -shared-libphobos -ldl execution test
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 Johannes Pfau changed: What|Removed |Added CC||code at dawg dot eu, ||johannespfau at gmail dot com --- Comment #8 from Johannes Pfau --- Regarding the _d_dso_registry issue: Yes, as far as I can see it is a bug that handleForName dlcloses the handle here. I think what happened is this: handleForName is used in one place with the comment // get handle without loading the library so it is supposed to unload the library there. But it is also called from handleForAddr which is used to get the DSO handle to be stored using setDSOForHandle. I think here, it's not really valid to store the closed handle. Let's ping the expert here though, I've added Martin Nowak to CC.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 Rainer Orth changed: What|Removed |Added URL||https://gcc.gnu.org/ml/gcc- ||patches/2019-01/msg01661.ht ||ml --- Comment #7 from Rainer Orth --- Patch posted, together with followups: https://gcc.gnu.org/ml/gcc-patches/2019-01/msg01663.html https://gcc.gnu.org/ml/gcc-patches/2019-01/msg01664.html
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #6 from ro at CeBiTec dot Uni-Bielefeld.DE --- The revised patch (still with loads of hacks) allowed the vast majority of Solaris 11.4/x86 gdc tests to PASS, both 32 and 64-bit: === gdc Summary === # of expected passes61178 # of unexpected failures36 # of unresolved testcases 12 The results are identical on Solaris 11.3/x86 with the patches for PR d/87864 (which provides minfo bracketing if ld doesn't) and https://gcc.gnu.org/ml/gcc-patches/2018-11/msg02248.html (which links with -lsocket -lnsl if need be). Solaris/SPARC is still another issue: I'll report my findings separately. Here are the changes from the previous version of the patch: * Solaris ld doesn't provide __bss_start (and no exact equivalent). Since __bss_start is declared weak in rt/bss_section.c, this won't get noticed at link time, but manifests itself as a SEGV caused by an assertion failure: Thread 2 received signal SIGSEGV, Segmentation fault. [Switching to Thread 1 (LWP 1)] gc_malloc (sz=40, ba=0, ti=0xff051250 ) at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/gc/proxy.d:117 117 return instance.malloc(sz, ba, ti); (gdb) where #0 gc_malloc (sz=40, ba=0, ti=0xff051250 ) at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/gc/proxy.d:117 #1 0xfefdd018 in _d_newclass ( ci=0xff051250 ) at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/rt/lifetime.d:96 #2 0xfefaa5cc in onAssertError (file=..., line=918) at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/core/exception.d:441 #3 0xfefaaedc in _d_assert (file=..., line=918) at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/core/exception.d:641 #4 0xfefe9ff4 in rt.sections_elf_shared.getCopyRelocSection() () at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/rt/sections_elf_shared.d:918 #5 0xfefebfbc in _d_dso_registry (data=0xffbfe288) at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/rt/sections_elf_shared.d:402 #6 0xfef968b8 in gdc.dso_ctor () at :1 #7 gdc.dso_ctor () at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/core/atomic.d:1 #8 global constructors keyed to 4core6atomic () at /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/core/atomic.d:1 #9 0xff1a0ccc in call_array () from /usr/lib/ld.so.1 #10 0xff1a0e4c in call_init () from /usr/lib/ld.so.1 #11 0xff1a015c in setup () from /usr/lib/ld.so.1 #12 0xff1b1d44 in _setup () from /usr/lib/ld.so.1 #13 0xff192738 in _rt_boot () from /usr/lib/ld.so.1 Backtrace stopped: previous frame identical to this frame (corrupt stack?) probably an assertion failure for findDSOInfoForAddr(bss_start, &info) || assert(0); since bss_start will be 0 if __bss_start doesn't exist. I've hacked around this by using _edata instead, which is close, but doesn't account for the alignment of the .bss section. * Solaris currently lacks dlpi_tls_modid. The field will probably be added to struct dl_phdr_info in Solaris 11.5 and perhaps backported to 11.4. I may be able to use the hack (making use of undocumented libc internals) from src/rt/sections_ldc.d in the ldc-developers druntime repo in on github if this turns out to work and be stable across Solaris 10 to 11.3. For now, I'm cheating by always setting _tlsMod to 0 on Solaris and disabling the corresponding assertions. * This allowed the gdc execution tests with -static-libphobos to PASS, but the -shared-libphobos tests would still FAIL, again with a SEGV Thread 2 received signal SIGSEGV, Segmentation fault. [Switching to Thread 1 (LWP 1)] 0xfe5a9559 in gc_malloc (sz=40, ba=0, ti=0xfe5eb180 ) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/gc/proxy.d:117 117 return instance.malloc(sz, ba, ti); (gdb) where #0 0xfe5a9559 in gc_malloc (sz=40, ba=0, ti=0xfe5eb180 ) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/gc/proxy.d:117 #1 0xfe553478 in core.memory.GC.malloc(uint, uint, const(TypeInfo)) (sz=40, ba=0, ti=0xfe5eb180 ) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/core/memory.d:380 #2 0xfe576a04 in _d_newclass ( ci=0xfe5eb180 ) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/rt/lifetime.d:96 #3 0xfe551994 in onAssertError (file=..., line=657) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/core/exception.d:441 #4 0xfe551d12 in _d_assert (file=..., line=657) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/core/exception.d:641 #5 0xfe583579 in rt.sections_elf_shared.setDSOForHandle(rt.sections_elf_shared.DSO*, void*) (pdso=0x8062420, handle=0xfe6d27a8) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/rt/sections_elf_shared.d:657 #6 0xfe5822f0 in _d_dso_registry (data=0xfeffd3d4) at /vol/gcc/src/hg/trunk/solaris/libphobos/libdruntime/rt/sections_elf_shared.d:409 #7 0xfe0c3540 in gdc.dso_ctor () at :1 #8 0xfe0c34b2 in global constructors keyed to 3etc1c4cu
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 Rainer Orth changed: What|Removed |Added Attachment #45067|0 |1 is obsolete|| --- Comment #5 from Rainer Orth --- Created attachment 45211 --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=45211&action=edit Revised patch
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #4 from Iain Buclaw --- (In reply to r...@cebitec.uni-bielefeld.de from comment #3) > > --- Comment #1 from Iain Buclaw --- > > Thanks, I will write up a small documentation of how the sections modules > > interacts with runtime - along with compiler support. > > Great, thanks. My current goal is to get anything to compile and link > in order to check if the runtime is any good on Solaris. > To give you an initial idea, of what sections is used for, I'll repost what I said to someone else when I started looking at OSX port. * ModuleInfo - All runtime module information are written out to a custom section. For statically linked applications, this section is scanned for module constructors, destructors, unittests, etc... * DSO support - builds on top of minfo with a _d_dso_registry() function that's called once when the dso is loaded, and once on unloading, passing the start and end address of this custom section, so the runtime can keep track of where all minfo are at runtime. * TLS - all static data is thread local by default, and the garbage collector needs to know where TLS data segment is located in all DSOs so that live memory isn't accidentally released. Emulated TLS makes this interesting. * Global data - same as TLS, but requires no special handling other than registering all rw data segments as GC roots. I might as well put this and then some as documentation in the sources.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #3 from ro at CeBiTec dot Uni-Bielefeld.DE --- > --- Comment #1 from Iain Buclaw --- > Thanks, I will write up a small documentation of how the sections modules > interacts with runtime - along with compiler support. Great, thanks. My current goal is to get anything to compile and link in order to check if the runtime is any good on Solaris. > As its heavily dependent on compiler and object format, I'll move the > implementation out of the common rt package and over into gcc also. I wouldn't expect the differences from other ELF targets to be too big, but we'll see...
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #2 from ro at CeBiTec dot Uni-Bielefeld.DE --- A couple of issues in the patch bear commenting: * The core/sys/solaris/dlfcn.d change was needed to silence /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/rt/sections_elf_shared.d:628:15: error: @nogc function 'rt.sections_elf_shared.linkMapForHandle' cannot call non-@nogc function 'core.sys.solaris.dlfcn.dlinfo' 628 | dlinfo(handle, RTLD_DI_LINKMAP, &map) == 0 || assert(0); | ^ /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/rt/sections_elf_shared.d:977:15: error: @nogc function 'rt.sections_elf_shared.handleForAddr' cannot call non-@nogc function 'core.sys.solaris.dlfcn.dladdr' 977 | if (dladdr(addr, &info) != 0) | ^ * The core/sys/solaris/sys/link.d change was required against /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/rt/sections_elf_shared.d:692:48: error: need 'this' for 'd_ptr' of type 'uint' 692 | strtab = cast(const(char)*)dyn.d_un.d_ptr; // FIXME: Which one? |^ /vol/gcc/src/hg/trunk/local/libphobos/libdruntime/rt/sections_elf_shared.d:705:34: error: need 'this' for 'd_val' of type 'uint' 705 | auto name = strtab + dyn.d_un.d_val; | ^ This didn't tell me anything and only after I'd closely compared the solaris and freebsd versions did I see the difference. * Enabling the compilation of rt/bss_section.c on Solaris was needed to get definitions for rt_get_bss_start and rt_get_end. I don't know what the upstream policy for linelength is, so I just use an overlong line here. * As the FIXME comment says, I don't know if the _Dmodule_ref in rt/sections_elf_shared.d is good for anything. I couldn't find any definition. * The Solaris struct dl_phdr_info doesn't have a dlpi_tls_modid member. For now, I've hacked around this to make things compile. 's rd_loadobj_t has a rt_tlsmodid member; maybe it's possible to use that. I'll check with the Solaris linker engineers if I get stuck. * Besides, dl_iterate_phdr was only introduced in some (late) Solaris 10 patch and only lives in libdl there, not libc as on Solaris 11. Enabling libphobos will have to check for that condition. * Right now, we have three identical versions of findDSOInfoForAddr. It may be useful to unify them. * Solaris 10 lacks getprogname; one could just return a hardcoded "a.out" instead as a fallback. * I don't know details about the files in core/sys/solaris: according to their comments they were produced on/for Illumos, and Oracle Solaris may well have deviated since. I'll have to compare them in detail at some point. No idea if they were handwritten or generated. * Solaris includes , , and as appropriate. It seems to me that core/sys/solaris/sys/elf.d should do the same.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 --- Comment #1 from Iain Buclaw --- Thanks, I will write up a small documentation of how the sections modules interacts with runtime - along with compiler support. As its heavily dependent on compiler and object format, I'll move the implementation out of the common rt package and over into gcc also.
[Bug d/88150] Use sections_elf_shared.d on Solaris
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88150 Rainer Orth changed: What|Removed |Added Target Milestone|--- |9.0