https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119588
--- Comment #4 from GCC Commits <cvs-commit at gcc dot gnu.org> --- The master branch has been updated by Matthew Malcomson <[email protected]>: https://gcc.gnu.org/g:901560aa377051f162bbf50a0fe49a5e17b4b232 commit r17-5886-g901560aa377051f162bbf50a0fe49a5e17b4b232 Author: Matthew Malcomson <[email protected]> Date: Wed Oct 7 16:38:11 2026 +0100 libgomp: Parallelise thread {re-,}initialisation This patch is a separate beneficial step of the work I'm doing towards PR119588. In that bug I describe how some internal workloads are hitting some performance problems w.r.t. the creation of many parallel regions one after each other. I'm attempting to reduce overhead of parallel region creation when using a large number of threads. Earlier work adjusts the barrier design to something that performs better under heavy contention. This is a patch built on top of that earlier work, but with appropriate rebase could be independent. It addresses a different area where I've seen a performance impact. It's the area that I mentioned in the below mail to the GCC mailing list. https://gcc.gnu.org/pipermail/gcc/2025-September/246618.html The fundamental point is that the cost of teardown & setup of a parallel region for LLVM is very close to the cost of a single barrier. For us there is a significant chunk of extra work in the actual teardown and setup. When re-using a team with no changes in various ICVs there is very little work that is required to be performed on top of a plain barrier. Even when some ICV's have changed a large amount of the work that is currently in a loop performed by the primary thread while all threads are waiting on it can be moved into secondary threads. This patch makes those changes. There are three distinct but related cases: 1) When re-using a thread unchanged. 2) When re-using a thread with some changes (or when leaving a thread to exit). 3) When starting a new thread. In all of these cases `gomp_team_start` currently initialises the same value into each threads local value for all of the below: - *thr->task (initialisation) - thr->num_teams - thr->team_num - fn (either thr->fn or data->fn) - data (either thr->data or data->fn_data) - thr->ts.def_allocator Hence each of these can be handled by writing this information to a shared location on the relevant team and having each thread read the information from there. There is some data that is always set to the same value (at least in relation to the team we're running on): - thr->ts.work_share - thr->ts.last_work_share - thr->ts.single_count - thr->ts.static_trip - thr->task (where it is located) Initialisation of these is directly moved from the primary thread to the secondary threads without any new communication being added. There are some variables that are always the same for a non-nested team and always equal for all threads of a nested team: - thr->ts.active_level - thr->ts.level These are communicated to secondary threads when starting a new team via the `start_data` of the new pthread, and are left as-is for the re-use of threads in a non-nested team. Then finally there are some member variables that are kept the same as for the last team for all reused threads of a non-nested team as long as the places/affinity ICV's are not changed (and the team is reused): - thr->ts.place_partition_off - thr->ts.place_partition_len - thr->place - thr->ts.team_id - thr->team - Which thread's `release` member address is stored in which `team->ordered_release[i]`. We maintain a cache of the last used ICV's in the `team` structure. If these match then we can completely skip the loop where the primary initialises parts of each secondary threads local data. These must be provided to any new secondary threads when they are starting. The last part to allow the primary thread to avoid iterating over all secondary threads when nothing has changed is to invert the protocol used to communicate that a given thread must exit. The existing convention is that secondary threads set their own `thr->fn` to `NULL`, then the primary sets the `nthr->fn` of each thread it wants to continue to the function that said secondary thread must execute. This means the primary thread needs to perform a write on each thread that is continuing. Here I switch the convention so that the primary thread sets `thr->exit` on each of the threads it wants to exit. Those threads without this boolean flag set read the function that they are to execute from their team structure. This means the primary thread writes once per exiting thread instead of once per reused thread. In the case where the number of threads and affinity setup is not changing between parallel regions this means no work instead of O(N) work. There is still a clause in the main loop of `gomp_thread_start` checking for `thr->fn` in order to let `gomp_pause_host` and `gomp_free_thread` continue to work as intended. I could have changed those functions to adjust `pool->last_team->ti.fn` instead of setting `thr->fn` directly but it didn't seem any better either way. Only subtlety here is that in order to ensure memory safety the secondary thread must avoid performing this initialisation when `thr->fn != NULL`. If a secondary non-nested thread gets "left to exit" when a new team is started the primary thread could race ahead, cache the new team, and free the team that this secondary thread was using all before the secondary thread goes through the new initialisation code. That could lead to a use-after-free. Hence we have to early-exit when this thread is not going to perform any more work. There is a synchronisation side-benefits to putting the initialisation in each secondary thread: - gomp_init_task is now guaranteed to be called on the thread-local task *after* gomp_finish_task is called on it. This avoids any leak of `depend_hash` members. (When team_id has not changed these functions are called on the same thread, when they have changed we add a new synchronisation point between new threads and threads that have been "remapped"). ------- Testing done: - Bootstrap & regtest on aarch64 and x86_64. - With & without _LIBGOMP_CHECKING_. - Testsuite with & without OMP_WAIT_POLICY=passive for GOMP_BARRIER_TYPE=flat, GOMP_BARRIER_TYPE=tree, GOMP_BARRIER_TYPE=centralized, and unspecified barrier type. - With and without configure `--enable-linux-futex=no` for posix target. - With and without configure `--enable-linux-futex-waitv=no` to check we haven't broken original linux/ target. - Cross compilation & regtest on arm. - Built gcn, rtems, nvptx targets (just building -- not ran testsuite). - TSAN with checks against new/old "fingerprints" of warnings raised from the libgomp testsuite. Performance testing done: - SPEC 2026 speed (those benchmarks using OpenMP) - SPEC OMP 2012 - SPEC HPC 2021 (restricted to one socket) - EPCC OpenMP Microbenchmarks. - Microbenchmark mentioned in PR119588. EPCC OpenMP microbenchmarks ran with the cross-product of: - OMP_WAIT_POLICY={active,unset,passive} - OMP_PROC_BIND={close,unset} - OMP_NUM_THREADS={4,unset} - GOMP_BARRIER_TYPE={flat,tree,centralized,auto} libgomp/ChangeLog: PR libgomp/119588 * libgomp.h (struct gomp_team): Add fields for communication at team start time. New `ti` member, new `ppv` member, new `reused` boolean. (struct gomp_thread): Add new `exit` boolean. * team.c (gomp_thread_start): Initialise thread-local data using members stored on team. (get_last_team): Set `team->reused` member when reusing. (gomp_new_team): Unset `team->reused` member when creating new team. (gomp_team_start): Primary thread stores data on team structure instead of copying that data to each secondary threads gomp_thread structure. Skip re-calculation of places on a reused team when inputs are the same. (gomp_init_team_state): New function. (gomp_thread_start_data): New `team_id`, `active_level`, `place_partition_off`, `place_partition_len`, and `team` members. Remove the `fn`, `fn_data`, `num_teams`, `team_num`, `ts`, and `task` members. Signed-off-by: Matthew Malcomson <[email protected]>
