cmcfarlen opened a new pull request, #13997: URL: https://github.com/apache/trafficserver/pull/13997
Every IOBuffer data block allocation and free does a `cmpxchg16b` on the global freelist head for its size class. perf c2c on an H2 cache-hit load at 32 event threads puts that line first: 7.55% of all HITMs. The 4K–32K classes carry most per-transaction buffer traffic, so this adds a per-thread cache of free blocks for those four sizes in front of `ioBufAllocator`, the same way `THREAD_ALLOC` caches other objects. - New record `proxy.config.allocator.iobuf_thread_cache_size`: bytes of free blocks per size class per thread. The default 512 KB gives 128/64/32/16 blocks for 4K/8K/16K/32K, so at most 2 MB per thread. Each class is capped at 512 blocks, and a class whose budget fits fewer than two blocks isn't cached. When a cache overflows, it returns blocks to the global freelist in one bulk free until a quarter of the limit is left. Set it to `0` to turn the caches off. - `Thread::~Thread` drains the caches. **History.** An unused `ProxyAllocator ioBufAllocator[]` was removed in `336bd0a675` (2014). #2995 (2018) proposed per-thread caching for `IOBufferData::_data`, and @bryancall raised two concerns: the gain was small, and memory could grow to threads × 2 MB × 512 × 2 because every size class up to 2 MB was cached. He suggested limiting which buffer sizes get cached. This does that: only 4K–32K are cached, each class is bounded by a byte budget (2 MB per thread in total with the default), and the cache can be turned off. The gain here is measured at 32 threads, where the freelist head is the most contended line. **Results** (H2 `/ip`, kTLS, 32 event threads, LTO Release, base = master @ `3767597`): | Mode | base req/s | with cache | load CPU µs/req | |---|---:|---:|---| | static, 32 × 192 (2 runs) | 503,254 / 501,296 | 538,356 / 538,928 (**+7.2%**) | 64.58 → 60.15 (−6.9%) | | cache hit, 32 × 192 (3 runs) | 918,589 mean | 925,590 mean (+0.8%, in noise) | 34.59 → 34.63 | In cache mode, the freelist head drops out of the top contended lines. The contention moves to the next shared atomics (remap table and config refcounts, global stats counters), which follow-up PRs address. **Downsides** - Blocks allocated on one thread and freed on another still go to the global freelist once per block on the allocating side. - Blocks sitting in thread caches show as in use in `ink_freelists_dump`. - As with the other thread caches, ASAN needs `-F` to see use-after-free on these blocks. Testing: new `test_IOBuffer` case covering watermark trimming, sizes that bypass the cache, a zero budget, and draining. Passes on Linux and macOS. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
