Hi hackers, This patch implements a suggestion from Andres' "Desired Changes" list [1] regarding the misleading nature of the WAL segment creation count in 'log_checkpoints' output. The "WAL file(s) added" field in checkpoint/restartpoint log output currently counts only segments added through PreallocXlogFiles(). It does not include segments created by other paths such as backend WAL growth, walreceiver, or timeline initialization, so on a write-heavy system the reported value can stay at 0 or 1 even when many new WAL segments were actually created.
This version fixes the accounting by adding a shared-memory atomic counter, walSegmentsCreated, to XLogCtlData. The counter is incremented in the WAL-segment creation/install paths, XLogFileInitInternal() and XLogFileCopy() (used for timeline switches). Each checkpoint or restartpoint reports WAL file(s) added as the difference between the current counter value and a saved baseline, walSegsCreatedLastCheckpoint, then advances that baseline. Because the baseline is initialized to 0, the first checkpoint after startup counts all segments created since startup, including timeline-initialization segments created before the end-of-recovery checkpoint is requested. The semantic change is intentional: 'ckpt_segs_added' (and arg2 of the TRACE_POSTGRESQL_CHECKPOINT_DONE probe) now means "new WAL segments created since the previous successful checkpoint or restartpoint, by any process." The probe arity is unchanged. config.sgml and monitoring.sgml are updated accordingly. Patch attached. Any feedback is welcome. [1]https://wiki.postgresql.org/wiki/User:Andresfreund/Desired_Changes#Add_information_about_the_number_of_newly_created_WAL_segments_to_log_checkpoints_output -- Best, Xuneng
From bdab62425cfaca7e12c19d51381cce9170932909 Mon Sep 17 00:00:00 2001 From: alterego655 <[email protected]> Date: Mon, 23 Mar 2026 14:53:39 +0800 Subject: [PATCH v1] Count WAL segment creations by all processes in log_checkpoints output The "WAL file(s) added" field in log_checkpoints output (and the corresponding ckpt_segs_added field reported via TRACE_POSTGRESQL_ CHECKPOINT_DONE) previously counted only segments preallocated by PreallocXlogFiles() inside CreateCheckPoint() and CreateRestartPoint(). Segments created by ordinary backends, the WAL receiver, or during end-of-recovery timeline initialization were silently excluded, making the reported count misleading on busy systems. Fix this by introducing a shared-memory atomic counter, walSegmentsCreated in XLogCtlData, which is incremented by any process whenever a new WAL segment file is installed via XLogFileInitInternal() or XLogFileCopy(). Each checkpoint or restartpoint computes ckpt_segs_added as the difference between the current counter value and a stored baseline, then advances the baseline. The baseline is initialized to zero, so the first checkpoint after startup naturally captures segments created during end-of-recovery timeline initialization. The metric is now "new WAL segment files created since the previous successful checkpoint or restartpoint, by any process." This is a deliberate semantic change from "preallocated by the checkpointer." The arity and types of TRACE_POSTGRESQL_CHECKPOINT_DONE are unchanged; only the meaning of arg2 changes. The old direct increment of CheckpointStats.ckpt_segs_added inside PreallocXlogFiles() is removed. Update config.sgml and monitoring.sgml to document the new semantics. --- doc/src/sgml/config.sgml | 4 +++ doc/src/sgml/monitoring.sgml | 10 ++++--- src/backend/access/transam/xlog.c | 47 +++++++++++++++++++++++++++++-- 3 files changed, 55 insertions(+), 6 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 8cdd826fbd3..8a20a34caec 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -7674,6 +7674,10 @@ local0.* /var/log/postgresql Causes checkpoints and restartpoints to be logged in the server log. Some statistics are included in the log messages, including the number of buffers written and the time spent writing them. + The number of WAL files added counts all new WAL segment files created + by any process (including regular backends and the WAL receiver) since + the previous checkpoint or restartpoint, not only those preallocated + by the checkpointer. This parameter can only be set in the <filename>postgresql.conf</filename> file or on the server command line. The default is on. </para> diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 462019a972c..0a5c84b3243 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -7690,10 +7690,12 @@ FROM pg_stat_get_backend_idset() AS backendid; <entry><literal>checkpoint-done</literal></entry> <entry><literal>(int, int, int, int, int)</literal></entry> <entry>Probe that fires when a checkpoint is complete. - (The probes listed next fire in sequence during checkpoint processing.) - arg0 is the number of buffers written. arg1 is the total number of - buffers. arg2, arg3 and arg4 contain the number of WAL files added, - removed and recycled respectively.</entry> + (The probes listed next fire in sequence during checkpoint processing.) + arg0 is the number of buffers written. arg1 is the total number of + buffers. arg2 is the number of new WAL files created by any process + since the previous checkpoint or restartpoint. + arg3 and arg4 contain the number of WAL files + removed and recycled respectively.</entry> </row> <row> <entry><literal>clog-checkpoint-start</literal></entry> diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f5c9a34374d..118eaf11656 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -475,6 +475,24 @@ typedef struct XLogCtlData pg_atomic_uint64 logWriteResult; /* last byte + 1 written out */ pg_atomic_uint64 logFlushResult; /* last byte + 1 flushed */ + /* + * Cumulative count of new WAL segment files created since startup, by any + * process. Used to compute per-checkpoint "WAL file(s) added" via + * differencing against walSegsCreatedLastCheckpoint. + */ + pg_atomic_uint64 walSegmentsCreated; + + /* + * Value of walSegmentsCreated recorded when the last checkpoint or + * restartpoint computed its ckpt_segs_added count. The next + * checkpoint/restartpoint diffs against this to get its own count, and + * then advances this value. Initialized to 0, so the first checkpoint + * captures all segments created since startup (including end-of-recovery + * timeline initialization). Only the checkpointer writes this; no lock + * required. + */ + uint64 walSegsCreatedLastCheckpoint; + /* * Latest initialized page in the cache (last byte position + 1). * @@ -3369,6 +3387,7 @@ XLogFileInitInternal(XLogSegNo logsegno, TimeLineID logtli, logtli)) { *added = true; + pg_atomic_fetch_add_u64(&XLogCtl->walSegmentsCreated, 1); elog(DEBUG2, "done creating and filling new WAL file"); } else @@ -3552,6 +3571,7 @@ XLogFileCopy(TimeLineID destTLI, XLogSegNo destsegno, */ if (!InstallXLogFileSegment(&destsegno, tmppath, false, 0, destTLI)) elog(ERROR, "InstallXLogFileSegment should not have failed"); + pg_atomic_fetch_add_u64(&XLogCtl->walSegmentsCreated, 1); } /* @@ -3727,8 +3747,6 @@ PreallocXlogFiles(XLogRecPtr endptr, TimeLineID tli) lf = XLogFileInitInternal(_logSegNo, tli, &added, path); if (lf >= 0) close(lf); - if (added) - CheckpointStats.ckpt_segs_added++; } } @@ -5093,6 +5111,7 @@ XLOGShmemInit(void) XLogCtl->SharedRecoveryState = RECOVERY_STATE_CRASH; XLogCtl->InstallXLogFileSegmentActive = false; XLogCtl->WalWriterSleeping = false; + pg_atomic_init_u64(&XLogCtl->walSegmentsCreated, 0); SpinLockInit(&XLogCtl->Insert.insertpos_lck); SpinLockInit(&XLogCtl->info_lck); @@ -7019,6 +7038,7 @@ CreateCheckPoint(int flags) VirtualTransactionId *vxids; int nvxids; int oldXLogAllowed = 0; + uint64 current; /* * An end-of-recovery checkpoint is really a shutdown checkpoint, just @@ -7473,6 +7493,18 @@ CreateCheckPoint(int flags) if (!RecoveryInProgress()) TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning()); + /* + * Compute the number of new WAL segments created since the last + * checkpoint or restartpoint (by any process), and advance the baseline + * for the next interval. The initial baseline is 0, so the first + * checkpoint captures segments created during end-of-recovery timeline + * initialization. + */ + current = pg_atomic_read_u64(&XLogCtl->walSegmentsCreated); + CheckpointStats.ckpt_segs_added = (int) + (current - XLogCtl->walSegsCreatedLastCheckpoint); + XLogCtl->walSegsCreatedLastCheckpoint = current; + /* Real work is done; log and update stats. */ LogCheckpointEnd(false, flags); @@ -7724,6 +7756,7 @@ CreateRestartPoint(int flags) XLogRecPtr endptr; XLogSegNo _logSegNo; TimestampTz xtime; + uint64 current; /* Concurrent checkpoint/restartpoint cannot happen */ Assert(!IsUnderPostmaster || MyBackendType == B_CHECKPOINTER); @@ -7944,6 +7977,16 @@ CreateRestartPoint(int flags) if (EnableHotStandby) TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning()); + /* + * Compute the number of new WAL segments created since the last + * checkpoint or restartpoint (by any process), and advance the baseline + * for the next interval. + */ + current = pg_atomic_read_u64(&XLogCtl->walSegmentsCreated); + CheckpointStats.ckpt_segs_added = (int) + (current - XLogCtl->walSegsCreatedLastCheckpoint); + XLogCtl->walSegsCreatedLastCheckpoint = current; + /* Real work is done; log and update stats. */ LogCheckpointEnd(true, flags); -- 2.51.0
