jimmystewpot opened a new pull request, #3752:
URL: https://github.com/apache/brpc/pull/3752

   ### What problem does this PR solve?
   
   Issue Number:
   
   Problem Summary:
   
   This issue was discovered while profiling StarRocks (which relies heavily on 
`brpc` / `bthread` / `butil` for RPC and concurrency) on ARM64 servers, where 
CPU profiles showed significant time and cache-coherency stalls spent inside 
legacy `ldxr`/`stxr` loops and unconditional `dmb ish` memory barriers.
   
   `src/butil/atomicops_internals_arm64_gcc.h` has not been updated since its 
original 2014 Chromium import and suffers from three key limitations on modern 
ARM64 (`__aarch64__`) platforms:
   
   1. **Inability to utilise ARMv8.1-A LSE or Outline Atomics:**
      `NoBarrier_CompareAndSwap`, `NoBarrier_AtomicExchange`, and 
`NoBarrier_AtomicIncrement` (both 32-bit and 64-bit) use literal `ldxr`/`stxr` 
load-linked/store-conditional assembly loops. Because these are opaque assembly 
strings to the compiler, neither GCC nor Clang can emit ARMv8.1-A Large System 
Extensions (LSE) instructions (`cas`, `swp`, `ldadd`, `stadd`) when targeting 
`-march=armv8-a+lse` / `-march=armv8.2-a+`, nor can they emit 
runtime-dispatched `-moutline-atomics` helper calls (`__aarch64_cas*`, 
`__aarch64_ldadd*`, `__aarch64_swp*`) on baseline `-march=armv8-a` builds.
   2. **Redundant Full-System Memory Barriers (`dmb ish`):**
      `Barrier_AtomicIncrement`, `Acquire_CompareAndSwap`, and 
`Release_CompareAndSwap` wrap relaxed LL/SC loops in full `MemoryBarrier()` 
(`dmb ish`) fences—even executing a full `dmb ish` barrier when 
`Acquire_CompareAndSwap` fails—rather than using architectural C++11/ARMv8 
acquire/release semantics (`ldaxr`/`stlxr` on ARMv8.0-A or 
`casa`/`casl`/`ldaddal` on ARMv8.1-A+ LSE).
   3. **Deprecated `<libkern/OSAtomic.h>` Fallback on Apple Silicon:**
      In `src/butil/atomicops.h`, `#elif defined(OS_MACOSX)` precedes `#elif 
defined(COMPILER_GCC) && defined(ARCH_CPU_ARM64)`, routing macOS ARM64 builds 
to the deprecated `OSAtomic*` routines in `butil/atomicops_internals_mac.h`.
   
   ### What is changed and the side effects?
   
   Changed:
   - **`src/butil/atomicops_internals_arm64_gcc.h`:** Replaced all 32-bit 
(`Atomic32`) and 64-bit (`Atomic64`) inline assembly blocks with GCC/Clang 
`__atomic_*` built-ins (`__atomic_compare_exchange_n`, `__atomic_exchange_n`, 
`__atomic_add_fetch`, `__atomic_store_n`, `__atomic_load_n`, and 
`__atomic_thread_fence`), preserving Chromium's `Acquire_Store` and 
`Release_Load` semantics.
   - **`src/butil/atomicops.h` & `BUILD.bazel`:** Ordered `#elif 
defined(COMPILER_GCC) && defined(ARCH_CPU_ARM64)` ahead of `#elif 
defined(OS_MACOSX)` so Apple Silicon builds use 
`atomicops_internals_arm64_gcc.h`, and exported the header in `macos_lib.hdrs` 
for Bazel builds on macOS.
   - **`test/atomicops_unittest.cc`:** Expanded unit test coverage to exercise 
`Barrier_AtomicIncrement`, `Acquire_CompareAndSwap`, `Release_CompareAndSwap` 
(both success and failure paths), explicit `Atomic64` instantiations, and 
multi-threaded CAS/increment contention.
   
   Side effects:
   - Performance effects:
     - **Disassembly (`objdump -d`):**
       - Baseline `-march=armv8-a` (GCC 13.3 / Clang 21): 0 raw `ldxr`/`stxr` 
instructions in caller object files; emits runtime-detected `-moutline-atomics` 
helper calls (`__aarch64_cas*`, `__aarch64_ldadd*`, `__aarch64_swp*`).
       - `-march=armv8-a+lse`: Emits inline `cas`, `casa`, `casl`, `swp`, 
`ldadd`, and `ldaddal` instructions with 0 `dmb ish` barriers in 
`Acquire_CompareAndSwap`, `Release_CompareAndSwap`, and 
`Barrier_AtomicIncrement`.
     - **Contention Benchmark (ARM Neoverse-V2, 4 threads contended, Ubuntu 
24.04, GCC 13.3.0):**
       | Benchmark (4 threads contended) | Before (`ldxr`/`stxr` + `dmb ish`) | 
After (`-march=armv8-a` outline LSE) | After (`-march=armv8-a+lse` inline LSE) |
       | :--- | :---: | :---: | :---: |
       | `Barrier_AtomicIncrement` | 46.34 ns/op | **15.39 ns/op** (3.01x 
faster) | **7.48 ns/op** (6.20x faster) |
       | Contended CAS Spinlock (`Acquire_CAS` + `Release_Store`) | 68.01 ns/op 
| **29.74 ns/op** (2.29x faster) | **32.71 ns/op** (2.08x faster) |
   
   - Breaking backward compatibility: No. Preserves 100% API and ABI 
compatibility for `butil::subtle::Atomic32`, `Atomic64`, and `AtomicWord`.
   
   ---
   ### Check List:
   - Please make sure your changes are compilable.
   - When providing us with a new feature, it is best to add related tests.
   - Please follow [Contributor Covenant Code of 
Conduct](https://github.com/apache/brpc/blob/master/CODE_OF_CONDUCT.md).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to