This is an automated email from the ASF dual-hosted git repository. xiaoxiang781216 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/nuttx.git
commit 29b6fce8e1e7f6d1d0f909cfd7baf54e3cb85731 Author: Marco Casaroli <[email protected]> AuthorDate: Mon Oct 19 11:41:57 2026 +0200 libs/libc/tls: Keep a copy of each __thread variable per thread. A toolchain without native TLS, such as the Arm GNU toolchain for aarch64-none-elf, compiles __thread variables into calls to __emutls_get_address(). That toolchain's libgcc is built with the single thread model, so its __emutls_get_address() keeps one copy of each variable for the whole program: all threads read and write the same copy. Add __wrap___emutls_get_address() and __wrap___emutls_register_common() to libc, and link the image with "--wrap=__emutls_get_address --wrap=__emutls_register_common" (tools/Config.mk and libs/libc/tls/CMakeLists.txt; CMake also wraps the user image of a protected build). The calls then go to libc whatever the order of the libraries, and the linker never takes libgcc's copy. With LTO the linker sees these calls only in the code made after it has chosen the archive members, so an LTO link also gets "-u" for the two __wrap_ names. EMUTLS_LDFLAGS gives the same options in the form for $(LD). The Toolchain.defs of each architecture that can have CONFIG_TLS_EMUTLS, and the sim Make.defs, add it to LDELFFLAGS and LDMODULEFLAGS, and the export keeps it, so the links of modules and programs get it too. The system symbol table (libc.csv) exports the __wrap_ names, which a module linked with --wrap refers to. The real names are weak aliases for a link without --wrap, such as the user image of a protected build or a program built outside NuttX: libc comes before libgcc there, and in a --whole-archive link libgcc's strong definitions win without a conflict. CONFIG_TLS_EMUTLS (default y) builds this for the GCC and clang toolchains when libgcc is the builtin library (BUILTIN_TOOLCHAIN; compiler-rt has its own copy per thread), but not for armclang or GHS. The sim has it with GCC and the GNU linker (Linux, and MinGW, whose GCC uses emulated TLS): the sim links NuttX into nuttx.rel with --wrap, and nuttx-names.in renames the weak real names, so the host program keeps its own libgcc copy. The macOS and MSVC linkers have no --wrap, so the sim there does not have it. The control object and the parameters use the types of libgcc's emutls.c, as GCC declares its built-in __emutls_register_common(): a machine word for the size and the alignment and an integer as wide as a pointer for the index, which is read and set with the atomic_ptr_t operations (16 bits on AVR, with the previous commit). Each variable gets a global index on its first use. Each thread keeps its copies in an array that its TLS info points to; a copy is made on the first use from the variable's template, or zeroed. exit(), quick_exit() and pthread_exit() free the copies after the exit functions and the TLS destructors, which may still use them. They call the code through a pointer that it sets on its first use, so a program without __thread variables does not link it. Without CONFIG_TLS_EMUTLS there is no pointer and no call. A thread that gets a new copy of the TLS info of its parent (vfork()) starts with no copies. Nothing depends on pthread keys or CONFIG_TLS_NELEM. With LTO every program links this code (about 800 bytes on Cortex-M; see the -u above), so the option has a prompt: nucleo-g431rb:nsh, the one LTO configuration without room for it, disables it. The --wrap options for ELF programs and modules (the sim Make.defs part of this commit) come to the CMake build in a separate change. [skip-cmake-parity] Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Marco Casaroli <[email protected]> --- CMakeLists.txt | 8 +- arch/arm/src/common/Toolchain.defs | 5 + arch/arm64/src/Toolchain.defs | 5 + arch/mips/src/mips32/Toolchain.defs | 5 + arch/misoc/src/lm32/Toolchain.defs | 5 + arch/misoc/src/minerva/Toolchain.defs | 5 + arch/risc-v/src/common/Toolchain.defs | 5 + arch/sim/src/nuttx-names.in | 4 + arch/sparc/src/sparc_v8/Toolchain.defs | 5 + arch/x86_64/src/common/Toolchain.defs | 5 + arch/xtensa/src/lx6/Toolchain.defs | 5 + arch/xtensa/src/lx7/Toolchain.defs | 5 + .../stm32g4/nucleo-g431rb/configs/nsh/defconfig | 1 + boards/sim/sim/sim/scripts/Make.defs | 5 + include/nuttx/tls.h | 28 +++ libs/libc/libc.csv | 2 + libs/libc/libc.h | 8 + libs/libc/pthread/pthread_exit.c | 11 + libs/libc/stdlib/lib_exit.c | 20 ++ libs/libc/tls/CMakeLists.txt | 24 ++ libs/libc/tls/Kconfig | 20 ++ libs/libc/tls/Make.defs | 4 + libs/libc/tls/tls_emutls.c | 265 +++++++++++++++++++++ libs/libc/tls/tls_getinfo.c | 12 + sched/tls/tls_dupinfo.c | 6 + tools/Config.mk | 27 +++ tools/Export.mk | 1 + tools/mkexport.sh | 2 +- 28 files changed, 494 insertions(+), 4 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 1d329dfa965..d4b48586b6b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -906,6 +906,7 @@ else() # renamed. The final nuttx binary is built by linking the host-specific # objects with the relocatable binary. include(nuttx_redefine_symbols) + get_property(nuttx_sim_rel_options GLOBAL PROPERTY NUTTX_SIM_REL_LINK_OPTIONS) # TODO: do with single function call? set(nuttx_libs_paths) @@ -928,9 +929,10 @@ else() OUTPUT nuttx.rel COMMAND ${CMAKE_C_COMPILER} ARGS -r $<$<BOOL:${CONFIG_SIM_M32}>:-m32> - $<$<BOOL:${LINUX}>:-Wl,-z,noexecstack> $<TARGET_OBJECTS:sim_head> - $<$<NOT:$<BOOL:${APPLE}>>:-Wl,--start-group> ${nuttx_libs_paths} - $<$<NOT:$<BOOL:${APPLE}>>:-Wl,--end-group> -o nuttx.rel + $<$<BOOL:${LINUX}>:-Wl,-z,noexecstack> ${nuttx_sim_rel_options} + $<TARGET_OBJECTS:sim_head> $<$<NOT:$<BOOL:${APPLE}>>:-Wl,--start-group> + ${nuttx_libs_paths} $<$<NOT:$<BOOL:${APPLE}>>:-Wl,--end-group> -o + nuttx.rel COMMAND ${CMAKE_OBJCOPY} --redefine-syms=nuttx-names.dat nuttx.rel DEPENDS ${nuttx_libs} sim_head sim_redefine_symbols COMMAND_EXPAND_LISTS) diff --git a/arch/arm/src/common/Toolchain.defs b/arch/arm/src/common/Toolchain.defs index 64534c2cd2b..824c7b5f0d4 100644 --- a/arch/arm/src/common/Toolchain.defs +++ b/arch/arm/src/common/Toolchain.defs @@ -758,6 +758,11 @@ endif LDELFFLAGS += -e _start -T $(call CONVERT_PATH,$(TOPDIR)$(DELIM)libs$(DELIM)libc$(DELIM)elf$(DELIM)gnu-elf.ld) +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) + # Zig toolchain include $(TOPDIR)/tools/Zig.defs diff --git a/arch/arm64/src/Toolchain.defs b/arch/arm64/src/Toolchain.defs index c36ba795098..41ee6442d6a 100644 --- a/arch/arm64/src/Toolchain.defs +++ b/arch/arm64/src/Toolchain.defs @@ -317,3 +317,8 @@ ifneq ($(CONFIG_BUILD_KERNEL),y) endif LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)$(DELIM)libs$(DELIM)libc$(DELIM)elf$(DELIM)gnu-elf.ld) + +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) diff --git a/arch/mips/src/mips32/Toolchain.defs b/arch/mips/src/mips32/Toolchain.defs index 4c1f0e325d7..99a6167a67d 100644 --- a/arch/mips/src/mips32/Toolchain.defs +++ b/arch/mips/src/mips32/Toolchain.defs @@ -338,3 +338,8 @@ CXXELFFLAGS += -fno-use-cxa-atexit LDELFFLAGS = -r -e _start LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)$(DELIM)libs$(DELIM)libc$(DELIM)elf$(DELIM)gnu-elf.ld) + +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) diff --git a/arch/misoc/src/lm32/Toolchain.defs b/arch/misoc/src/lm32/Toolchain.defs index 431f901444e..84d550c80c6 100644 --- a/arch/misoc/src/lm32/Toolchain.defs +++ b/arch/misoc/src/lm32/Toolchain.defs @@ -152,3 +152,8 @@ CXXELFFLAGS += -fno-use-cxa-atexit LDELFFLAGS = -r -e _start LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)$(DELIM)libs$(DELIM)libc$(DELIM)elf$(DELIM)gnu-elf.ld) + +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) diff --git a/arch/misoc/src/minerva/Toolchain.defs b/arch/misoc/src/minerva/Toolchain.defs index 12d2e3afc4a..c756244c6f0 100644 --- a/arch/misoc/src/minerva/Toolchain.defs +++ b/arch/misoc/src/minerva/Toolchain.defs @@ -100,3 +100,8 @@ CXXELFFLAGS += -fno-use-cxa-atexit LDELFFLAGS = -r -e _start LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)$(DELIM)libs$(DELIM)libc$(DELIM)elf$(DELIM)gnu-elf.ld) + +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) diff --git a/arch/risc-v/src/common/Toolchain.defs b/arch/risc-v/src/common/Toolchain.defs index dca6c97a0ba..519637c43c8 100644 --- a/arch/risc-v/src/common/Toolchain.defs +++ b/arch/risc-v/src/common/Toolchain.defs @@ -503,6 +503,11 @@ endif LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)/libs/libc/elf/gnu-elf.ld) +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) + # Zig toolchain include $(TOPDIR)/tools/Zig.defs diff --git a/arch/sim/src/nuttx-names.in b/arch/sim/src/nuttx-names.in index 9449f9e33f2..0a2f663ddb5 100644 --- a/arch/sim/src/nuttx-names.in +++ b/arch/sim/src/nuttx-names.in @@ -33,6 +33,10 @@ #endif NXSYMBOLS(__cxa_atexit) +#ifdef CONFIG_TLS_EMUTLS +NXSYMBOLS(__emutls_get_address) +NXSYMBOLS(__emutls_register_common) +#endif NXSYMBOLS(abort) NXSYMBOLS(accept) NXSYMBOLS(access) diff --git a/arch/sparc/src/sparc_v8/Toolchain.defs b/arch/sparc/src/sparc_v8/Toolchain.defs index 03bf6d8e0c6..449761f8a18 100644 --- a/arch/sparc/src/sparc_v8/Toolchain.defs +++ b/arch/sparc/src/sparc_v8/Toolchain.defs @@ -143,3 +143,8 @@ CXXELFFLAGS += -fno-use-cxa-atexit LDELFFLAGS = -r -e _start LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)$(DELIM)libs$(DELIM)libc$(DELIM)elf$(DELIM)gnu-elf.ld) + +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) diff --git a/arch/x86_64/src/common/Toolchain.defs b/arch/x86_64/src/common/Toolchain.defs index f61ddb99dc4..514382af4a9 100644 --- a/arch/x86_64/src/common/Toolchain.defs +++ b/arch/x86_64/src/common/Toolchain.defs @@ -300,6 +300,11 @@ endif LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)$(DELIM)libs$(DELIM)libc$(DELIM)elf$(DELIM)gnu-elf.ld) +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) + # -fno-pic to avoid GOT relocations # -mcmodel=large to avoid out-of-range 32-bit relocations diff --git a/arch/xtensa/src/lx6/Toolchain.defs b/arch/xtensa/src/lx6/Toolchain.defs index d0f290e47f7..61544610710 100644 --- a/arch/xtensa/src/lx6/Toolchain.defs +++ b/arch/xtensa/src/lx6/Toolchain.defs @@ -248,3 +248,8 @@ ifneq ($(CONFIG_BUILD_KERNEL),y) LDELFFLAGS += $(TOPDIR)$(DELIM)arch$(DELIM)xtensa$(DELIM)src$(DELIM)crt0.o endif + +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) diff --git a/arch/xtensa/src/lx7/Toolchain.defs b/arch/xtensa/src/lx7/Toolchain.defs index d94bf3868b6..09b4b5c9305 100644 --- a/arch/xtensa/src/lx7/Toolchain.defs +++ b/arch/xtensa/src/lx7/Toolchain.defs @@ -261,3 +261,8 @@ ifneq ($(CONFIG_BUILD_KERNEL),y) LDELFFLAGS += $(TOPDIR)$(DELIM)arch$(DELIM)xtensa$(DELIM)src$(DELIM)crt0.o endif + +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) diff --git a/boards/arm/stm32g4/nucleo-g431rb/configs/nsh/defconfig b/boards/arm/stm32g4/nucleo-g431rb/configs/nsh/defconfig index f80f9cbba38..4f5e601644d 100644 --- a/boards/arm/stm32g4/nucleo-g431rb/configs/nsh/defconfig +++ b/boards/arm/stm32g4/nucleo-g431rb/configs/nsh/defconfig @@ -11,6 +11,7 @@ # CONFIG_NSH_DISABLE_IFCONFIG is not set # CONFIG_NSH_DISABLE_PS is not set # CONFIG_SYSTEM_DD is not set +# CONFIG_TLS_EMUTLS is not set CONFIG_ARCH="arm" CONFIG_ARCH_BOARD="nucleo-g431rb" CONFIG_ARCH_BOARD_COMMON=y diff --git a/boards/sim/sim/sim/scripts/Make.defs b/boards/sim/sim/sim/scripts/Make.defs index 6fb7ae1e8cb..5062c87ff14 100644 --- a/boards/sim/sim/sim/scripts/Make.defs +++ b/boards/sim/sim/sim/scripts/Make.defs @@ -304,6 +304,11 @@ endif LDELFFLAGS = -r -e main --gc-sections LDELFFLAGS += -T $(call CONVERT_PATH,$(TOPDIR)/libs/libc/elf/gnu-elf.ld) +# The --wrap options of CONFIG_TLS_EMUTLS (tools/Config.mk) + +LDMODULEFLAGS += $(EMUTLS_LDFLAGS) +LDELFFLAGS += $(EMUTLS_LDFLAGS) + ifeq ($(CONFIG_HOST_MACOS),y) LDFLAGS += -Wl,-dead_strip else diff --git a/include/nuttx/tls.h b/include/nuttx/tls.h index 7cf9c077c21..d14c0897a98 100644 --- a/include/nuttx/tls.h +++ b/include/nuttx/tls.h @@ -98,6 +98,15 @@ extern "C" typedef CODE void (*tls_dtor_t)(FAR void *); +#ifdef CONFIG_TLS_EMUTLS +/* The integer types of the emulated TLS of libgcc: a machine word, and an + * integer as wide as a pointer. + */ + +typedef unsigned int emutls_word_t __attribute__((mode(word))); +typedef unsigned int emutls_pointer_t __attribute__((mode(pointer))); +#endif + /* This structure encapsulates all variables associated with getopt(). */ struct getopt_s @@ -286,6 +295,9 @@ struct tls_info_s int tl_errno; /* Per-thread error number */ pid_t tl_tid; /* Thread ID */ FAR char **tl_argv; /* Arguments first string */ +#ifdef CONFIG_TLS_EMUTLS + FAR void *tl_emutls; /* Copies of __thread variables */ +#endif /* Robust mutex support ***************************************************/ @@ -421,6 +433,22 @@ FAR struct tls_info_s *tls_get_info(void); void tls_destruct(void); #endif +/**************************************************************************** + * Name: __wrap___emutls_get_address and __wrap___emutls_register_common + * + * Description: + * The per-thread emulated TLS of libc. --wrap sends the calls that GCC + * makes to __emutls_get_address() and __emutls_register_common() here. + * The system symbol table (libc.csv) needs these prototypes. + * + ****************************************************************************/ + +#ifdef CONFIG_TLS_EMUTLS +FAR void *__wrap___emutls_get_address(FAR void *arg); +void __wrap___emutls_register_common(FAR void *arg, emutls_word_t size, + emutls_word_t align, FAR void *templ); +#endif + /**************************************************************************** * Name: task_get_info * diff --git a/libs/libc/libc.csv b/libs/libc/libc.csv index db795f17132..0f58865bd8a 100644 --- a/libs/libc/libc.csv +++ b/libs/libc/libc.csv @@ -9,6 +9,8 @@ "__gcov_reset","gcov.h","defined(CONFIG_COVERAGE_MINI) && defined(CONFIG_ARCH_TOOLCHAIN_GCC)","void" "__res_state","resolv.h","defined(CONFIG_NETDB_DNSCLIENT)","FAR struct __res_state *" "__stack_chk_fail","ssp/ssp.h","defined(CONFIG_STACK_CANARIES)","void","void" +"__wrap___emutls_get_address","nuttx/tls.h","defined(CONFIG_TLS_EMUTLS)","FAR void *","FAR void *" +"__wrap___emutls_register_common","nuttx/tls.h","defined(CONFIG_TLS_EMUTLS)","void","FAR void *","emutls_word_t","emutls_word_t","FAR void *" "_alert","debug.h","!defined(CONFIG_CPP_HAVE_VARARGS) && defined(CONFIG_DEBUG_ERROR)","void","FAR const char *","..." "_err","debug.h","!defined(CONFIG_CPP_HAVE_VARARGS) && defined(CONFIG_DEBUG_ERROR)","void","FAR const char *","..." "_exit","unistd.h","","void","int" diff --git a/libs/libc/libc.h b/libs/libc/libc.h index 6a1465a1530..1e289c06260 100644 --- a/libs/libc/libc.h +++ b/libs/libc/libc.h @@ -328,6 +328,14 @@ ssize_t lib_parse_hostfile(FAR FILE *stream, FAR struct hostent *host, int lib_restoredir(void); #endif +/* Defined in tls/tls_getinfo.c. tls/tls_emutls.c sets it, and exit() + * and pthread_exit() call it. + */ + +#ifdef CONFIG_TLS_EMUTLS +extern CODE void (*g_tls_emutls_destruct)(void); +#endif + /* Defined in lib_cxx_initialize.c */ #ifdef CONFIG_HAVE_CXXINITIALIZE diff --git a/libs/libc/pthread/pthread_exit.c b/libs/libc/pthread/pthread_exit.c index 1facdc57036..c8008699316 100644 --- a/libs/libc/pthread/pthread_exit.c +++ b/libs/libc/pthread/pthread_exit.c @@ -33,6 +33,8 @@ #include <nuttx/pthread.h> #include <nuttx/tls.h> +#include "libc.h" + /**************************************************************************** * Public Functions ****************************************************************************/ @@ -68,6 +70,15 @@ void pthread_exit(FAR void *exit_value) tls_destruct(); #endif +#ifdef CONFIG_TLS_EMUTLS + /* Then free the copies of its __thread variables */ + + if (g_tls_emutls_destruct != NULL) + { + g_tls_emutls_destruct(); + } +#endif + nx_pthread_exit(exit_value); PANIC(); } diff --git a/libs/libc/stdlib/lib_exit.c b/libs/libc/stdlib/lib_exit.c index 5eb310dec2d..7c9cc374eae 100644 --- a/libs/libc/stdlib/lib_exit.c +++ b/libs/libc/stdlib/lib_exit.c @@ -37,6 +37,8 @@ #include <nuttx/pthread.h> #include <nuttx/lib/elf.h> +#include "libc.h" + #ifndef __KERNEL__ /**************************************************************************** @@ -111,6 +113,15 @@ void exit(int status) tls_destruct(); #endif +#ifdef CONFIG_TLS_EMUTLS + /* Then free the copies of its __thread variables */ + + if (g_tls_emutls_destruct != NULL) + { + g_tls_emutls_destruct(); + } +#endif + #if defined(CONFIG_LIBC_DLFCN) && defined(CONFIG_BUILD_PROTECTED) /* Close what dlopen() left open. The kernel cannot reach this registry. * A flat build does it in the kernel, after the last thread. A kernel @@ -174,6 +185,15 @@ void quick_exit(int status) tls_destruct(); #endif +#ifdef CONFIG_TLS_EMUTLS + /* Then free the copies of its __thread variables */ + + if (g_tls_emutls_destruct != NULL) + { + g_tls_emutls_destruct(); + } +#endif + /* Then perform the exit */ _exit(status); diff --git a/libs/libc/tls/CMakeLists.txt b/libs/libc/tls/CMakeLists.txt index 70dfba3d7b2..9aa26d6c37a 100644 --- a/libs/libc/tls/CMakeLists.txt +++ b/libs/libc/tls/CMakeLists.txt @@ -22,6 +22,30 @@ set(SRCS task_getinfo.c tls_getinfo.c) +# Send the calls to __emutls_get_address() to the per-thread version of libc +# (see tools/Config.mk). + +if(CONFIG_TLS_EMUTLS) + list(APPEND SRCS tls_emutls.c) + set(EMUTLS_WRAP -Wl,--wrap=__emutls_get_address + -Wl,--wrap=__emutls_register_common) + # with LTO the calls appear only after the linker has chosen the archive + # members: take the libc functions in from the start + if(NOT CONFIG_LTO_NONE) + list(APPEND EMUTLS_WRAP -Wl,-u,__wrap___emutls_get_address + -Wl,-u,__wrap___emutls_register_common) + endif() + if(CONFIG_ARCH_SIM) + # the sim links NuttX into nuttx.rel first (see CMakeLists.txt) + set_property(GLOBAL PROPERTY NUTTX_SIM_REL_LINK_OPTIONS ${EMUTLS_WRAP}) + else() + target_link_options(nuttx PRIVATE ${EMUTLS_WRAP}) + endif() + if(CONFIG_BUILD_PROTECTED) + target_link_options(nuttx_user PRIVATE ${EMUTLS_WRAP}) + endif() +endif() + if(NOT CONFIG_TLS_TASK_NELEM EQUAL 0) list(APPEND SRCS task_tls.c task_tls_destruct.c) endif() diff --git a/libs/libc/tls/Kconfig b/libs/libc/tls/Kconfig index f91222dc2d9..6b9b9533fb2 100644 --- a/libs/libc/tls/Kconfig +++ b/libs/libc/tls/Kconfig @@ -5,6 +5,26 @@ menu "Thread Local Storage (TLS)" +config TLS_EMUTLS + bool "Per-thread copies of emulated TLS variables" + default y + depends on BUILTIN_TOOLCHAIN + depends on (ARCH_TOOLCHAIN_GNU && !ARCH_TOOLCHAIN_ARMCLANG && !ARCH_TOOLCHAIN_GHS && !ARCH_SIM) || (ARCH_SIM && SIM_TOOLCHAIN_GCC && !HOST_MACOS && !WINDOWS_NATIVE) + ---help--- + GCC and clang call __emutls_get_address() for __thread variables + when the target has no native TLS, or with -femulated-tls. libc + then gives one copy of each variable per thread, and the link + sends the calls to it with --wrap. libgcc's copy is for one + thread only; compiler-rt (BUILTIN_COMPILER_RT) has its own copy + per thread. The sim has it only with GCC and the GNU linker + (Linux, MinGW): the macOS linker and the MSVC linker have no + --wrap. + + With LTO every program links this code (about 800 bytes on + Cortex-M). A board that has no space for it can disable it: then + all threads share one copy of each __thread variable, as with + libgcc. + config TLS_ALIGNED bool "Require stack alignment" default BUILD_KERNEL || BUILD_PROTECTED diff --git a/libs/libc/tls/Make.defs b/libs/libc/tls/Make.defs index fd109f59fd8..62065096ae3 100644 --- a/libs/libc/tls/Make.defs +++ b/libs/libc/tls/Make.defs @@ -22,6 +22,10 @@ CSRCS += task_getinfo.c tls_getinfo.c +ifeq ($(CONFIG_TLS_EMUTLS),y) +CSRCS += tls_emutls.c +endif + ifneq ($(CONFIG_TLS_TASK_NELEM),0) CSRCS += task_tls.c task_tls_destruct.c endif diff --git a/libs/libc/tls/tls_emutls.c b/libs/libc/tls/tls_emutls.c new file mode 100644 index 00000000000..cafbeb6007c --- /dev/null +++ b/libs/libc/tls/tls_emutls.c @@ -0,0 +1,265 @@ +/**************************************************************************** + * libs/libc/tls/tls_emutls.c + * + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. The + * ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the + * License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include <nuttx/config.h> + +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <nuttx/atomic.h> +#include <nuttx/mutex.h> +#include <nuttx/tls.h> + +#include "libc.h" + +/**************************************************************************** + * Private Types + ****************************************************************************/ + +/* The control object that GCC emits for a __thread variable when the + * target has no native TLS, with the types of libgcc's emutls.c. + */ + +struct __emutls_object +{ + emutls_word_t size; + emutls_word_t align; + union + { + emutls_pointer_t offset; + FAR void *ptr; + } loc; + FAR void *templ; +}; + +/* The copies of the variables that one thread has used. The TLS info of + * the thread points to it. + */ + +struct emutls_array_s +{ + uintptr_t size; + FAR void *data[1]; +}; + +/**************************************************************************** + * Private Data + ****************************************************************************/ + +/* The index of a variable is global: in a flat build all task groups share + * this data, but each variable has its own index. + */ + +static mutex_t g_emutls_lock = NXMUTEX_INITIALIZER; +static uintptr_t g_emutls_count; + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: emutls_alloc + * + * Description: + * Allocate the copy of a variable for the calling thread, initialized + * from the template of the variable, or zeroed if it has none. + * + ****************************************************************************/ + +static FAR void *emutls_alloc(FAR struct __emutls_object *obj) +{ + size_t align = obj->align > sizeof(FAR void *) ? + obj->align : sizeof(FAR void *); + FAR void *ptr; + + ptr = lib_memalign(align, obj->size); + if (ptr == NULL) + { + abort(); + } + + if (obj->templ != NULL) + { + memcpy(ptr, obj->templ, obj->size); + } + else + { + memset(ptr, 0, obj->size); + } + + return ptr; +} + +/**************************************************************************** + * Name: tls_emutls_destruct + * + * Description: + * Free the copies of the calling thread when it exits. + * + ****************************************************************************/ + +static void tls_emutls_destruct(void) +{ + FAR struct tls_info_s *tls = tls_get_info(); + FAR struct emutls_array_s *array = tls->tl_emutls; + uintptr_t i; + + if (array == NULL) + { + return; + } + + tls->tl_emutls = NULL; + for (i = 0; i < array->size; i++) + { + lib_free(array->data[i]); + } + + lib_free(array); +} + +/**************************************************************************** + * Name: emutls_getslot + * + * Description: + * Return the entry for index in the array of the calling thread. + * + ****************************************************************************/ + +static FAR void **emutls_getslot(FAR struct tls_info_s *tls, + uintptr_t index) +{ + FAR struct emutls_array_s *array = tls->tl_emutls; + FAR struct emutls_array_s *newarray; + uintptr_t oldsize = array != NULL ? array->size : 0; + uintptr_t newsize; + + if (index < oldsize) + { + return &array->data[index]; + } + + newsize = index + 16; + newarray = lib_realloc(array, sizeof(struct emutls_array_s) + + (newsize - 1) * sizeof(FAR void *)); + if (newarray == NULL) + { + abort(); + } + + memset(&newarray->data[oldsize], 0, + (newsize - oldsize) * sizeof(FAR void *)); + newarray->size = newsize; + tls->tl_emutls = newarray; + g_tls_emutls_destruct = tls_emutls_destruct; + return &newarray->data[index]; +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: __wrap___emutls_get_address + * + * Description: + * Return the address of the calling thread's copy of a __thread + * variable. libgcc's __emutls_get_address() keeps a single copy when + * libgcc is built without thread support; --wrap sends the calls here. + * + ****************************************************************************/ + +FAR void *__wrap___emutls_get_address(FAR void *arg) +{ + FAR struct __emutls_object *obj = arg; + FAR void **slot; + uintptr_t index; + + /* loc.offset is the index plus one, or 0 before the first use */ + + index = atomic_ptr_read_acquire(&obj->loc.offset); + if (index == 0) + { + nxmutex_lock(&g_emutls_lock); + index = obj->loc.offset; + if (index == 0) + { + index = ++g_emutls_count; + atomic_ptr_set_release(&obj->loc.offset, index); + } + + nxmutex_unlock(&g_emutls_lock); + } + + index--; + slot = emutls_getslot(tls_get_info(), index); + if (*slot == NULL) + { + *slot = emutls_alloc(obj); + } + + return *slot; +} + +/**************************************************************************** + * Name: __wrap___emutls_register_common + * + * Description: + * Merge the size, alignment and template of a common __thread variable + * that more than one object file defines. + * + ****************************************************************************/ + +void __wrap___emutls_register_common(FAR void *arg, emutls_word_t size, + emutls_word_t align, FAR void *templ) +{ + FAR struct __emutls_object *obj = arg; + + if (obj->size < size) + { + obj->size = size; + obj->templ = NULL; + } + + if (obj->align < align) + { + obj->align = align; + } + + if (templ != NULL && size == obj->size) + { + obj->templ = templ; + } +} + +/* A link without --wrap, such as the user image of a protected build or a + * program built outside NuttX, uses these names. libc comes before libgcc + * there, and libgcc's strong definitions win in a --whole-archive link. + */ + +weak_alias(__wrap___emutls_get_address, __emutls_get_address) +weak_alias(__wrap___emutls_register_common, __emutls_register_common) diff --git a/libs/libc/tls/tls_getinfo.c b/libs/libc/tls/tls_getinfo.c index a1d1480c3b8..759c40bf96a 100644 --- a/libs/libc/tls/tls_getinfo.c +++ b/libs/libc/tls/tls_getinfo.c @@ -32,6 +32,18 @@ #include <nuttx/tls.h> #include <nuttx/sched.h> +#include "libc.h" + +/**************************************************************************** + * Public Data + ****************************************************************************/ + +/* tls_emutls.c sets it on its first use, so exit() does not link it */ + +#ifdef CONFIG_TLS_EMUTLS +CODE void (*g_tls_emutls_destruct)(void); +#endif + /**************************************************************************** * Public Functions ****************************************************************************/ diff --git a/sched/tls/tls_dupinfo.c b/sched/tls/tls_dupinfo.c index 5ee29d93de9..e71185b87e1 100644 --- a/sched/tls/tls_dupinfo.c +++ b/sched/tls/tls_dupinfo.c @@ -77,6 +77,12 @@ int tls_dup_info(FAR struct tcb_s *dst, FAR struct tcb_s *src) info->tl_argv = NULL; +#ifdef CONFIG_TLS_EMUTLS + /* The parent's copies of the __thread variables stay with the parent */ + + info->tl_emutls = NULL; +#endif + /* Thread ID */ info->tl_tid = dst->pid; diff --git a/tools/Config.mk b/tools/Config.mk index 5ca7ea636bd..af9454a5bd2 100644 --- a/tools/Config.mk +++ b/tools/Config.mk @@ -914,3 +914,30 @@ LOWER_CASE = $(call ULMAP,$(LOWERMAP),$(1)) ifeq ($(CONFIG_ITERABLE_SECTIONS_LINKER_INSERT),y) ARCHSCRIPT += $(TOPDIR)$(DELIM)include$(DELIM)nuttx$(DELIM)linker$(DELIM)common-insert.ld endif + +# Send the calls to __emutls_get_address() to the per-thread version of +# libc, never to the single copy of libgcc. LDFLAGS is for the image link; +# the sim links NuttX with "$(LD) -r $(LDLINKFLAGS)" first. The arch +# Toolchain.defs adds EMUTLS_LDFLAGS to LDELFFLAGS and LDMODULEFLAGS. + +ifeq ($(CONFIG_TLS_EMUTLS),y) + EMUTLS_WRAP = --wrap=__emutls_get_address --wrap=__emutls_register_common + EMUTLS_IMAGE = $(EMUTLS_WRAP) + + # With LTO the linker sees the calls only in the code made after it has + # chosen the archive members, so take the libc functions in from the start. + + ifneq ($(CONFIG_LTO_NONE),y) + EMUTLS_IMAGE += -u __wrap___emutls_get_address + EMUTLS_IMAGE += -u __wrap___emutls_register_common + endif + + ifeq ($(CONFIG_ARCH_SIM),y) + LDLINKFLAGS += $(EMUTLS_IMAGE) + else + LDFLAGS += $(EMUTLS_IMAGE) + endif + + EMUTLS_XLINKER = $(if $(filter $(CC),$(LD)),-Xlinker) + EMUTLS_LDFLAGS = $(foreach f,$(EMUTLS_WRAP),$(EMUTLS_XLINKER) $(f)) +endif diff --git a/tools/Export.mk b/tools/Export.mk index ca3187c0b3a..ea16468187e 100644 --- a/tools/Export.mk +++ b/tools/Export.mk @@ -53,6 +53,7 @@ $(EXPORTDIR)/makeinfo.sh: $(TOPDIR)/.config $(EXPORTDIR)/Make.defs @echo "CPP=\"$(CPP)\"" >> $(EXPORTDIR)/makeinfo.sh @echo "CROSSDEV=\"$(CROSSDEV)\"" >> $(EXPORTDIR)/makeinfo.sh @echo "CXX=\"$(CXX)\"" >> $(EXPORTDIR)/makeinfo.sh + @echo "EMUTLS_LDFLAGS=\"$(EMUTLS_LDFLAGS)\"" >> $(EXPORTDIR)/makeinfo.sh @echo "EXEEXT=\"$(EXEEXT)\"" >> $(EXPORTDIR)/makeinfo.sh @echo "EXTRA_LIBPATHS=\"$(EXTRA_LIBPATHS)\"" >> $(EXPORTDIR)/makeinfo.sh @echo "EXTRA_LIBS=\"$(EXTRA_LIBS)\"" >> $(EXPORTDIR)/makeinfo.sh diff --git a/tools/mkexport.sh b/tools/mkexport.sh index 894b55d0dc3..dea4433c25e 100755 --- a/tools/mkexport.sh +++ b/tools/mkexport.sh @@ -241,7 +241,7 @@ LDELFFLAGS=$(echo "$LDELFFLAGS" | sed -e 's:-T *[^ ]*\.ld::g') # Set LDMODULEFLAGS so that kernel modules can build in kernel mode -LDMODULEFLAGS="-r" +LDMODULEFLAGS="-r ${EMUTLS_LDFLAGS}" # Save the compilation options
