Hi, On Thu, Oct 29, 2026 at 03:58:03PM -0700, Lukas Fittl wrote: > Hi, > > Over in [0] we were recently discussing how parallel workers > accumulate their buffer usage into the leader and how that makes it > hard to pass pgBufferUsage instrumentation into the cumulative stats > system, and I've also been looking into this for the stack-based > instrumentation work [1], because it's really hard to avoid double > counting due to how we accumulate into the leader's > pgBufferUsage/pgWalUsage today. > > Turns out there is a current bug in master in how pg_stat_wal and > pg_stat_get_backend_wal() report activity from parallel workers. This > issue has existed since the introduction of pg_stat_wal in Postgres > 14: We double count WAL produced in parallel workers, because we add > WAL activity from the parallel workers to both the worker's pgWalUsage > (direct) and the leader's pgWalUsage (via InstrAccumParallelQuery), > and each of them emits that activity to the cumulative stats.
Ouch, nice catch! > In terms of how to fix this, I first tried to invent a mechanism that > would keep instrumentation imported from workers separate in the > leader (so we never double count in the global). But that turns out to > be pretty tricky, because we have use cases (pg_stat_statements being > the main one, but e.g. also EXPLAIN ANALYZE of a query calling a > function that itself runs a parallel query) where we want to track > whole-statement activity (leader + workers), including utility > statements, and we want to be able to do so in a nested manner. Agreed. > Instead, at least for now, the easier solution seems to be to > accumulate imported instrumentation data into both the existing > pg(Wal|Buffer)Usage globals, and a separate set called > pg(Wal|Buffer)UsageFromWorkers that counts only the activity from the > workers. Any caller that wants to isolate the leader-only activity for > reporting to the cumulative statistics system then subtracts the > worker portion from the global instrumentation using a helper function > (ComputeOwnWalUsage), and workers continue to report their own > activity directly to the cumulative statistics system. Yeah that looks like the best way to proceed currently. I wonder if the extra additions could be reduced though. What about first accumulating all workers into a local BufferUsage/WalUsage and then calling InstrAccumParallelQuery() once for all workers? Also, worth to add some tests? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com
