Hello Chenhui, I tried to find the reason why the preferred cap in the benchmark differs so much between our machines, and I came up with a theory that explains the behavior.
On 04.09.26 15:51, Jan Nidzwetzki wrote: > The preferred cap on the J5005 is the first one at or above this > threshold (0x100000 = 1 MB). Does this match the behavior on your > systems as well? glibc tunable ============= On x86, glibc switches memcpy() to non-temporal stores above x86_non_temporal_threshold (bypassing the CPU caches and writing data directly to the system memory). This value is dynamically calculated based on the characteristics of the machine. However, it can be adjusted manually, like: GLIBC_TUNABLES="glibc.cpu.x86_non_temporal_threshold=$value\ :glibc.cpu.x86_memset_non_temporal_threshold=$value" I did an experiment (outside of PostgreSQL) where I forced certain thresholds and repeated the benchmark. Changing that value moves the preferred cap with it. On the J5005, with a 256 MB output: set threshold 1 KB source 100 B source 64 KB cap64k cap64k 256 KB cap256k cap256k 1 MB cap1m cap1m 4 MB none none disabled none none The speedup always starts at the first cap at or above the threshold, when non-temporal stores are used and the cache is bypassed. At a 4 MB threshold, the effect disappears entirely. A 4 MB block is the whole L2 on this machine, so no size is both large enough for non-temporal stores and small enough to stay cached. With the non-temporal path disabled, the row is flat as well, and uncapped doubling falls below the current implementation. Since this is adjustable through GLIBC_TUNABLES, the best cap is not only machine-dependent; it is settable from the environment. We could potentially use something like __builtin_nontemporal_store() to avoid the glibc threshold and force non-temporal stores. However, this is compiler- and architecture-specific, and I don't think it is worth optimizing this particular code path that way. What do you think about the theory? Personally, I would go with the early return and memset() optimization as outlined in my last mail, since they show clear improvements in all tested environments. The doubling / capping approach shows good results in many situations, but is harder to implement, since picking a wrong cap leads to regressions. However, I am still curious what other people will see when they repeat the benchmarks. Best regards Jan -- Jan Nidzwetzki PlanetScale Postgres Core Team
