andygrove commented on PR #5934: URL: https://github.com/apache/datafusion-comet/pull/5934#issuecomment-5681578940
Here is the `alloc_overhead` microbenchmark with accounting off and on, plus the parallel case around the flush threshold that was asked for (added in 2d78f87 as `threshold_churn`). Before the numbers, one correction to the benchmark itself, fixed in ca2f8c8. In edition 2021 an `--extern` crate that nothing names is dropped from the crate graph. The accounting-off run of this benchmark named nothing in `comet`, so the rlib and its `#[global_allocator]` were never linked and the "jemalloc" baseline was in fact glibc malloc (the binary was 4 MB and contained no jemalloc symbols; with the feature on it names `comet::alloc_accounting` and is 28 MB with jemalloc linked). The bench now has an `extern crate comet` and a jemalloc liveness assertion next to the existing accounting one, so a run against the wrong allocator fails instead of producing a plausible number. The comparison below is jemalloc on both sides. **Setup:** woody, Ryzen 9 7950X3D (16 cores / 32 threads, one socket), Linux, `--features jemalloc` vs `--features jemalloc,alloc-accounting`, Criterion with 2 s warm-up and 4 s measurement. Single-thread cases are pinned to one core with `taskset` because this CPU has two chiplets with different cache and unpinned single-thread numbers swing by 2x between runs; a pinned off-vs-off rerun agrees within 2%. Parallel cases use all 32 threads. | case | off (jemalloc) | on (jemalloc + accounting) | delta | |---|---|---|---| | `alloc_free_16b` | 4.11 ns | 6.05 ns | +1.9 ns | | `alloc_free_256b` | 4.26 ns | 6.20 ns | +1.9 ns | | `alloc_free_4096b` | 6.10 ns | 7.89 ns | +1.8 ns | | `alloc_free_32kb` (never flushes) | 23.0 ns | 25.1 ns | +2.1 ns | | `alloc_free_64kb` (flushes on every alloc and free) | 200.9 ns | 204.5 ns | +3.6 ns | | `alloc_fill_free_64kb` | 616 ns | 611 ns | none (p = 0.56) | | `grow_vec_to_64kb` | 41.56 µs | 41.38 µs | none | | `parallel_alloc_free_32kb_x32` (never flushes) | 47.6 ns | 49.0 ns | +1.4 ns (p = 0.24) | | `parallel_alloc_free_64kb_x32` (every thread flushes on every call) | 295 ns | 320 ns | +25 ns (+8%) | Times are per alloc/free pair, and for the parallel rows per pair per thread, so a parallel number equal to its single-thread counterpart would mean no interference at all. **Reading it:** - The thread-local path costs about 2 ns per alloc/free pair, or 1 ns per call, independent of size. That is the whole cost when a thread's drift stays under 64 KiB, which is the 32 KiB rows: the alloc and the free cancel in the thread-local cell and the shared counter is never touched, single-threaded or on 32 threads. - A flush is two uncontended atomic read-modify-writes and adds another 1.5 ns on top of that single-threaded (64 KiB row). 64 KiB blocks are above jemalloc's thread-cache limit, so the allocator's own cost dominates at 200 ns. - Contention is the last row: 32 threads each doing two atomic adds on the same cacheline per iteration, with nothing else in between, cost 25 ns per pair, or 8% over jemalloc's own contended large-allocation path. That is the upper bound for the shared counter, and it needs every core to do nothing but allocate and free exactly-threshold blocks. Anything below the threshold never reaches the counter, and anything doing real work between allocations amortizes it. - The two rows closest to what Comet actually does, filling a batch-sized buffer and growing a builder, show no measurable difference. This is consistent with the 1.4% on the TPC-H SF100 sum of medians reported above, where the executors spend a small fraction of their time in the allocator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
