On Thu, Sep 1, 2016 at 3:29 PM, Nadav Har'El <[email protected]> wrote: > > On Thu, Sep 1, 2016 at 6:21 PM, David Crawshaw <[email protected]> wrote: >> >> I've just spent the last two days in the same code porting the runtime >> to a similarly almost-linux-ELF system, so I may have some information >> that's useful to you. There are several variants on how Go does this >> depending on the system, so please bear with me if I slip up. > > > Thanks for you answer. It was very informative. > >> >> >> If we fix GOOS=linux, GOARCH=amd64, and -buildmode=exe, then: >> >> Go follows the ELF ABI for TLS. It uses a single TLS slot, named >> runtime.tlsg. It's at -8. >> >> With internal linking, this fixed slot is easy to ensure. All go code >> only generates the one. The only external C that's linked in is >> trusted not to take a slot away. (Notice there's an exception where >> cmd/link refuses to internally link the C variant of the net package >> on dragonfly, because dragonfly uses __thread for errno, meaning the >> libc might want that slot.) >> >> With external linking, runtime.tlsg is the only TLS LE variable. All >> the others have to be at least LD or GD, because they are coming from >> shared libraries. > > > OSv does not currently play well with the local-exec TLS model (the reason > is explained > by pdziepak in https://github.com/cloudius-systems/osv/issues/352), but it > does work > well with the initial-exec model, so I am happy that some of the build modes > do use > initial-exec model (I also thought this was the case by looking at the > relocations > actually generate).
Yes, Go (linux/amd64) uses TLS IE for runtime.tlsg when -shared is passed to the compiler. This is done when -buildmode=c-shared. (Any use of TLS IE requires external linking, because cmd/link doesn't know how to generate the appropriate dynamic relocation for it.) The toolchain also uses -shared for -buildmode=pie, however in this mode -pie is passed to the external linker, which will optimize the TLS IE variable to LE. >> When CGO_ENABLED=0, there's a problem when new threads are created. >> Typically the libc creating the thread sets %fs to point to the TLS >> slots. But it's unset, because we created the thread without libc, via >> a direct clone syscall. So we need to emulate what the libc does, >> which we do in the runtime.settls function by calling arch_prctl. In >> particular we subtract 8 from the value we write into %fs so we get >> exactly what the libc would have done, we are not stealing the slot >> directly. In fact, after this is done, it's safe for that thread to >> run C code. Note that this function is *not* called when >> CGO_ENABLED=1. Hopefully the fact we are not clobbering %fs for this >> combination of build options makes your life easier. > > > I'm happy to hear both that the arch_prctl is meant to restore C's idea of > fs_base, and not to overwrite it with something else - and that there is an > option not to do it at all. This might mean that we don't need to support > generic arch_prctl in OSv at all (and all the mess required to save and > restore fs_base as we move between "user space" and "kernel" - a boundary > which doesn't really exist as such in OSv). > > I'm not familiar with the Go terminology - what does > "CGO_ENABLED=1" refer to? Is the "c-shared" build mode using this option? CGO_ENABLED is an environment variable, and controls whether cgo support is enabled. When it is (=1), you can use import "C". The build modes c-shared and c-archive imply cgo. It sounds like you'll need to use external linking if you need runtime.tlsg to be TLS IE. Using c-shared is reasonable if you have some machinery to do the equivalent of dlopen and call the global initializer the runtime inserts into the c-shared library. Does OSv support the equivalent of dynamically loading a .so that uses TLS IE? If so, buildmode=c-shared is a good way to go. >> >> All of these facts change as you vary GOOS, GOARCH, buildmode, and >> CGO_ENABLED. (For example, -buildmode=pie implies PIC code which makes >> the TLS variable IE instead of LE, darwin/android use far off fixed >> slot offsets, ARM has a more accessible TLS register, etc.) The >> trickiest thing around is CanUse1InsnTLS in the compiler, which >> tripped me up for a while. Spend some time reading the comment at the >> call site of that function. > > > I spent a couple of minutes reading it, can't say I understood it very well > - but > I'm also not sure if it's 100% relevant to what I'm trying to understand. Probably not, if you stick to linux/amd64. > By the way, on a whim, Benoit (CC'ed) tried the Go test case on OSv again > on a newer Go - golang-1.7.0 instead of the older 1.6.3 we used previously - > and it seems the TLS stuff (when using the c-shared build mode, I believe) > is > suddenly working as expected. Is it possible that TLS was recently fixed in > golang, and we were simply testing a broken version? Lots changed. I can't think of anything relevant, but there could well be some important difference. > Thanks again for taking the time to answer, > > Nadav. > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
