On Thu, Sep 1, 2016 at 6:21 PM, David Crawshaw <[email protected]> wrote:
> I've just spent the last two days in the same code porting the runtime > to a similarly almost-linux-ELF system, so I may have some information > that's useful to you. There are several variants on how Go does this > depending on the system, so please bear with me if I slip up. > Thanks for you answer. It was very informative. > > If we fix GOOS=linux, GOARCH=amd64, and -buildmode=exe, then: > > Go follows the ELF ABI for TLS. It uses a single TLS slot, named > runtime.tlsg. It's at -8. > > With internal linking, this fixed slot is easy to ensure. All go code > only generates the one. The only external C that's linked in is > trusted not to take a slot away. (Notice there's an exception where > cmd/link refuses to internally link the C variant of the net package > on dragonfly, because dragonfly uses __thread for errno, meaning the > libc might want that slot.) > > With external linking, runtime.tlsg is the only TLS LE variable. All > the others have to be at least LD or GD, because they are coming from > shared libraries. > OSv does not currently play well with the local-exec TLS model (the reason is explained by pdziepak in https://github.com/cloudius-systems/osv/issues/352), but it does work well with the initial-exec model, so I am happy that some of the build modes do use initial-exec model (I also thought this was the case by looking at the relocations actually generate). When CGO_ENABLED=0, there's a problem when new threads are created. > Typically the libc creating the thread sets %fs to point to the TLS > slots. But it's unset, because we created the thread without libc, via > a direct clone syscall. So we need to emulate what the libc does, > which we do in the runtime.settls function by calling arch_prctl. In > particular we subtract 8 from the value we write into %fs so we get > exactly what the libc would have done, we are not stealing the slot > directly. In fact, after this is done, it's safe for that thread to > run C code. Note that this function is *not* called when > CGO_ENABLED=1. Hopefully the fact we are not clobbering %fs for this > combination of build options makes your life easier. > I'm happy to hear both that the arch_prctl is meant to restore C's idea of fs_base, and not to overwrite it with something else - and that there is an option not to do it at all. This might mean that we don't need to support generic arch_prctl in OSv at all (and all the mess required to save and restore fs_base as we move between "user space" and "kernel" - a boundary which doesn't really exist as such in OSv). I'm not familiar with the Go terminology - what does "CGO_ENABLED=1" refer to? Is the "c-shared" build mode using this option? > All of these facts change as you vary GOOS, GOARCH, buildmode, and > CGO_ENABLED. (For example, -buildmode=pie implies PIC code which makes > the TLS variable IE instead of LE, darwin/android use far off fixed > slot offsets, ARM has a more accessible TLS register, etc.) The > trickiest thing around is CanUse1InsnTLS in the compiler, which > tripped me up for a while. Spend some time reading the comment at the > call site of that function. > I spent a couple of minutes reading it, can't say I understood it very well - but I'm also not sure if it's 100% relevant to what I'm trying to understand. By the way, on a whim, Benoit (CC'ed) tried the Go test case on OSv again on a newer Go - golang-1.7.0 instead of the older 1.6.3 we used previously - and it seems the TLS stuff (when using the c-shared build mode, I believe) is suddenly working as expected. Is it possible that TLS was recently fixed in golang, and we were simply testing a broken version? Thanks again for taking the time to answer, Nadav. -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
