Hi, A couple days back I happened to notice a blog post [1] by Jeremy Schneider, describing how recovery_prefetch can trigger an outage. I happened to be involved in getting that feature into PG 15, so I was curious how it could trigger such failures.
After discussing the details with Jeremy over the weekend, I believe it's really a combination of two issues. 1) recovery_prefetch fails to skip prefetches after an FPI, which may significantly increase the number of IO operations, to the point that the replica runs into disk limits etc. This inflates the replay lag. 2) replication behavior after walreceiver exit - In streaming mode, syncrep waits only for confirmation the WAL was flushed, so it works (mostly) OK even if the replay can't keep up. But if the walreceiver disconnects, the replication startup starts to apply WAL - which may be much slower due to the FPI amplification described in (1). Only when that apply of all the received WAL completes will it start streaming again, and confirming commits on the primary. AFAICS the real issue is (2), but it's not great the prefetching can regress the replay like this. That seems unfortunate even ignoring the syncrep aspect, as it may increase replay lag on read replicas etc. As pointed out in the blog post, Stephen mentioned FPIs [2] back in 2021, but we've all been concerned mostly about syscall overhead and buffer lookups. And the conclusion was that it's fine for v1, negligible relative to the benefits, and that if it turns out to be measurable we'll improve it. But as Jeremy demonstrates in his blog post, the actual issue is in issuing unnecessary I/O, saturating the device IOPS limit. In the reproducer, this is done by UPDATE pgbench_accounts SET abalance = abalance + 1 WHERE aid % 20 = 0 which effectively updates all pages of the table - multiple tuples on each. The first modification generates an FPI, and then there's a bunch of "non-FPI" changes. The prefetcher skips the FPI, but then still issues prefetches for that block due to the non-FPI changes. In reality no reads are necessary - replaying the FPI will install the buffer. The blog claims 16x IOPS amplification. I believe that's simply due to the UPDATE benefiting from read-ahead on the primary (reading data in 128K chunks), but standby dealing with 8K pages (because that's what we get in WAL). That's the same volume, but 16x more IOPS. There's a tiny cache of recently prefetched blocks, but it can't really help in this case - FPIs are not added to it, and even if they were it's way too small (just 4 entries). I kinda suspect it's not large enough in general, but that's a separate question. We actually got a report [3] describing this issue about 18 months ago, including a patch leveraging the XLOGPREFETCHER_SEQ_WINDOW_SIZE bit I mentioned above. That probably works for the reported example, but it's not really a proper fix - the default size "4" is way too low for FPIs. But I used it as a starting point, and after a bit of Sunday hacking, I got what I think is a much more complete fix. I'm attaching the whole series, as it's easier to understand the changes like that, but in the end it'd be squashed into a single commit IO think. I tried to explain the changes in commit messages, so only a very brief overview here: 0001-Avoid-useless-prefetches-in-case-of-recent-FPI-WA.patch --------------------------------------------------------------- Pretty much Michael's patch [3] from 2025, no other changes. 0002-Separate-the-FPIs-from-recently-prefetched-blocks.patch --------------------------------------------------------------- Separates the FPI tracking from recent_block, into a separate array with more entries (but still small, with linear search). 0003-Rework-the-FPI-cache-as-a-set-associative-cache.patch --------------------------------------------------------------- Replace the array / linear search with a set-associative cache, inspired by the cache used in fast-path locking. It seems perfect, with minimal overhead, and it's OK to handle overflows by just forgetting old FPIs. 0004-Move-FPI-tracking-into-separate-functions.patch --------------------------------------------------------------- Refactoring, moving code into functions to improve readability. Should squash into 0003. 0005-Use-a-proper-hash-function-for-FPI-history.patch --------------------------------------------------------------- Replace a stupid "hash function" mapping blocks to the set-associative cache with a much better one. Should squash into 0003. 0006-Ignore-replayed-FPIs-in-prefetch-FPI-history.patch --------------------------------------------------------------- At some point I realized the FPI entries become irrelevant once we replay the record. We have the LSN already, and we can just treat the entry as empty if the LSN is before replaing_lsn. It works more or less as an eviction. This way we won't have "stale entries", causing synchronous reads. 0007-Avoid-duplicate-entries-in-FPI-history.patch --------------------------------------------------------------- There's no point in tracking multiple FPIs for the same block, it's enough to track the new one. It's not very common, of course. But it can happen, and it's not more expensive. 0008-Size-recovery-FPI-history-by-the-look-ahead-dista.patch --------------------------------------------------------------- Until now the size of the FPI history was hard-coded, but we can size it based on the prefetch distance, to minimize "premature eviction". The cache is larger (up to ~400kB for maintenance_io_concurrency=1000), but there can only be one such cache in the walreceiver (and instance). Realistic values are a fraction of that. 0009-Skip-FPI-lookups-when-no-FPIs-are-pending.patch --------------------------------------------------------------- An optimization - a quick exit when there are no outstanding FPIs (which can happen close to the end of a checkpoint, etc). regards [1] https://ardentperf.com/2026/10/20/postgres-recovery_prefetch-considered-harmful-and-causing-outages/ [2] https://www.postgresql.org/message-id/[email protected] [3] https://www.postgresql.org/message-id/9bdb428a-fc8e-4801-8fff-1ad0bc6fa76b%40postgrespro.ru -- Tomas Vondra
From 65dde757ee4845747644221b9ce25273a82666b1 Mon Sep 17 00:00:00 2001 From: Michael Zhilin <[email protected]> Date: Sun, 30 Mar 2025 01:30:37 +0300 Subject: [PATCH v1 1/9] Avoid useless prefetches in case of recent FPI WAL records If block is logged into WAL with full page, then the startup process reads full page image from WAL stream and load them into shared buffer by XLogReadBufferForRedoExtended. Every further WAL records related to particular blocks should be skipped by XLogPrefetcher. Author: Michael Zhilin <[email protected]> Discussion: https://www.postgresql.org/message-id/9bdb428a-fc8e-4801-8fff-1ad0bc6fa76b%40postgrespro.ru --- src/backend/access/transam/xlogprefetcher.c | 22 ++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 288cdbd34c4..41c1caf45e1 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -54,7 +54,7 @@ /* * To detect repeated access to the same block and skip useless extra system - * calls, we remember a small window of recently prefetched blocks. + * calls, we remember a small window of recently prefetched / FPW blocks. */ #define XLOGPREFETCHER_SEQ_WINDOW_SIZE 4 @@ -679,6 +679,26 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) if (block->has_image) { XLogPrefetchIncrement(&SharedStats->skip_fpw); + + /* + * A full page image is copied to shared buffer during replay. + * Every next WAL records related to particular block doesn't + * require prefetching of shared buffer. + */ + for (int i = 0; i < XLOGPREFETCHER_SEQ_WINDOW_SIZE; ++i) + { + if (block->blkno == prefetcher->recent_block[i] && + RelFileLocatorEquals(block->rlocator, prefetcher->recent_rlocator[i])) + { + return LRQ_NEXT_NO_IO; + } + } + + prefetcher->recent_rlocator[prefetcher->recent_idx] = block->rlocator; + prefetcher->recent_block[prefetcher->recent_idx] = block->blkno; + prefetcher->recent_idx = + (prefetcher->recent_idx + 1) % XLOGPREFETCHER_SEQ_WINDOW_SIZE; + return LRQ_NEXT_NO_IO; } -- 2.55.0
From 0f825212eca0a1466cabccd9f1b02f903a18411a Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 12:03:34 +0100 Subject: [PATCH v1 2/9] Separate the FPIs from recently prefetched blocks Don't build the FPI skipping on the detection of recently prefetched blocks. We're not prefetching blocks from records with FPIs, so adding them to the recent_block cache is a bit misleading. It's also not clear why these two caches should use the same capacity, i.e. maybe 4 entries is enough for the recent_block cache, but we probably want a different (much larger) cache for the FPIs. Add a separate (larger) cache for FPIs, and check it after the existing recent_block cache. Extend the pg_stat_recovery_prefetch view to also track the number of prefetches skipped because of a recent FPI. Document the new field. XXX: The skip_fpi name is not very self-explanatory, and it's easy to confuse with skip_fpw. We should probably pick a better name. --- doc/src/sgml/monitoring.sgml | 11 +++ src/backend/access/transam/xlogprefetcher.c | 77 +++++++++++++++------ src/backend/catalog/system_views.sql | 1 + src/include/catalog/pg_proc.dat | 6 +- src/test/regress/expected/rules.out | 3 +- 5 files changed, 72 insertions(+), 26 deletions(-) diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 31ff4a9d99f..af8f9f5e336 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -2353,6 +2353,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage </entry> </row> + <row> + <entry role="catalog_table_entry"> + <para role="column_definition"> + <structfield>skip_fpi</structfield> <type>bigint</type> + </para> + <para> + Number of blocks not prefetched because they would be restored from a full page image included in an earlier WAL record + </para> + </entry> + </row> + <row> <entry role="catalog_table_entry"> <para role="column_definition"> diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 41c1caf45e1..ed06c3e429f 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -54,10 +54,19 @@ /* * To detect repeated access to the same block and skip useless extra system - * calls, we remember a small window of recently prefetched / FPW blocks. + * calls, we remember a small window of recently prefetched blocks. */ #define XLOGPREFETCHER_SEQ_WINDOW_SIZE 4 +/* + * A small history of most recently seen FPIs, so that we can skip useless + * prefetches for these blocks (most likely still in cache). + * + * XXX At which point should we switch to a hash table, instead of a linear + * search in a small array? + */ +#define XLOGPREFETCHER_FPI_HISTORY_SIZE 64 + /* * When maintenance_io_concurrency is not saturated, we're prepared to look * ahead up to N times that number of block references. @@ -143,6 +152,11 @@ struct XLogPrefetcher BlockNumber recent_block[XLOGPREFETCHER_SEQ_WINDOW_SIZE]; int recent_idx; + /* Book-keeping to avoid prefetching right after FPI */ + RelFileLocator recent_fpi_rlocator[XLOGPREFETCHER_FPI_HISTORY_SIZE]; + BlockNumber recent_fpi_block[XLOGPREFETCHER_FPI_HISTORY_SIZE]; + int recent_fpi_idx; + /* Book-keeping to disable prefetching temporarily. */ XLogRecPtr no_readahead_until; @@ -179,6 +193,7 @@ typedef struct XLogPrefetchStats pg_atomic_uint64 skip_init; /* Zero-inited blocks skipped. */ pg_atomic_uint64 skip_new; /* New/missing blocks filtered. */ pg_atomic_uint64 skip_fpw; /* FPWs skipped. */ + pg_atomic_uint64 skip_fpi; /* Accesses after FPI skipped. */ pg_atomic_uint64 skip_rep; /* Repeat accesses skipped. */ /* Dynamic values */ @@ -319,6 +334,7 @@ XLogPrefetchShmemInit(void *arg) pg_atomic_init_u64(&SharedStats->skip_init, 0); pg_atomic_init_u64(&SharedStats->skip_new, 0); pg_atomic_init_u64(&SharedStats->skip_fpw, 0); + pg_atomic_init_u64(&SharedStats->skip_fpi, 0); pg_atomic_init_u64(&SharedStats->skip_rep, 0); } @@ -334,6 +350,7 @@ XLogPrefetchResetStats(void) pg_atomic_write_u64(&SharedStats->skip_init, 0); pg_atomic_write_u64(&SharedStats->skip_new, 0); pg_atomic_write_u64(&SharedStats->skip_fpw, 0); + pg_atomic_write_u64(&SharedStats->skip_fpi, 0); pg_atomic_write_u64(&SharedStats->skip_rep, 0); } @@ -681,23 +698,17 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) XLogPrefetchIncrement(&SharedStats->skip_fpw); /* - * A full page image is copied to shared buffer during replay. - * Every next WAL records related to particular block doesn't - * require prefetching of shared buffer. + * We should not really see a FPI for the same block very + * often, not more than once per checkpoint cycle. So don't + * bother with scanning the history here. If we end up with a + * duplicate FPI, it means the system is almost idle. We also + * don't bother with evicting blocks from the history, for the + * same reason. */ - for (int i = 0; i < XLOGPREFETCHER_SEQ_WINDOW_SIZE; ++i) - { - if (block->blkno == prefetcher->recent_block[i] && - RelFileLocatorEquals(block->rlocator, prefetcher->recent_rlocator[i])) - { - return LRQ_NEXT_NO_IO; - } - } - - prefetcher->recent_rlocator[prefetcher->recent_idx] = block->rlocator; - prefetcher->recent_block[prefetcher->recent_idx] = block->blkno; - prefetcher->recent_idx = - (prefetcher->recent_idx + 1) % XLOGPREFETCHER_SEQ_WINDOW_SIZE; + prefetcher->recent_fpi_rlocator[prefetcher->recent_fpi_idx] = block->rlocator; + prefetcher->recent_fpi_block[prefetcher->recent_fpi_idx] = block->blkno; + prefetcher->recent_fpi_idx = + (prefetcher->recent_fpi_idx + 1) % XLOGPREFETCHER_FPI_HISTORY_SIZE; return LRQ_NEXT_NO_IO; } @@ -731,6 +742,27 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) return LRQ_NEXT_NO_IO; } } + + /* + * Likewise, no point in prefetching a block right after restoring + * it from an FPI. We keep a bit longer history of FPIs. + * + * XXX Maybe we could skip this with full-page writes disabled? + * + * XXX Maybe we could skip this backwards, with the assumption + * that if there is a FPI, it's likely very close? + */ + for (int i = 0; i < XLOGPREFETCHER_FPI_HISTORY_SIZE; ++i) + { + if (block->blkno == prefetcher->recent_fpi_block[i] && + RelFileLocatorEquals(block->rlocator, prefetcher->recent_fpi_rlocator[i])) + { + XLogPrefetchIncrement(&SharedStats->skip_fpi); + return LRQ_NEXT_NO_IO; + } + } + + /* We're going to prefetch the block, so add it to history. */ prefetcher->recent_rlocator[prefetcher->recent_idx] = block->rlocator; prefetcher->recent_block[prefetcher->recent_idx] = block->blkno; prefetcher->recent_idx = @@ -849,7 +881,7 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) Datum pg_stat_get_recovery_prefetch(PG_FUNCTION_ARGS) { -#define PG_STAT_GET_RECOVERY_PREFETCH_COLS 10 +#define PG_STAT_GET_RECOVERY_PREFETCH_COLS 11 ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; Datum values[PG_STAT_GET_RECOVERY_PREFETCH_COLS]; bool nulls[PG_STAT_GET_RECOVERY_PREFETCH_COLS]; @@ -865,10 +897,11 @@ pg_stat_get_recovery_prefetch(PG_FUNCTION_ARGS) values[3] = Int64GetDatum(pg_atomic_read_u64(&SharedStats->skip_init)); values[4] = Int64GetDatum(pg_atomic_read_u64(&SharedStats->skip_new)); values[5] = Int64GetDatum(pg_atomic_read_u64(&SharedStats->skip_fpw)); - values[6] = Int64GetDatum(pg_atomic_read_u64(&SharedStats->skip_rep)); - values[7] = Int32GetDatum(SharedStats->wal_distance); - values[8] = Int32GetDatum(SharedStats->block_distance); - values[9] = Int32GetDatum(SharedStats->io_depth); + values[6] = Int64GetDatum(pg_atomic_read_u64(&SharedStats->skip_fpi)); + values[7] = Int64GetDatum(pg_atomic_read_u64(&SharedStats->skip_rep)); + values[8] = Int32GetDatum(SharedStats->wal_distance); + values[9] = Int32GetDatum(SharedStats->block_distance); + values[10] = Int32GetDatum(SharedStats->io_depth); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); return (Datum) 0; diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index ddf801307eb..83fb070a617 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -1054,6 +1054,7 @@ CREATE VIEW pg_stat_recovery_prefetch AS s.skip_init, s.skip_new, s.skip_fpw, + s.skip_fpi, s.skip_rep, s.wal_distance, s.block_distance, diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 1f21a8eb6d4..ff1a233d5c6 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6106,9 +6106,9 @@ { oid => '6248', descr => 'statistics: information about WAL prefetching', proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{timestamptz,int8,int8,int8,int8,int8,int8,int4,int4,int4}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o}', - proargnames => '{stats_reset,prefetch,hit,skip_init,skip_new,skip_fpw,skip_rep,wal_distance,block_distance,io_depth}', + proallargtypes => '{timestamptz,int8,int8,int8,int8,int8,int8,int8,int4,int4,int4}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{stats_reset,prefetch,hit,skip_init,skip_new,skip_fpw,skip_fpi,skip_rep,wal_distance,block_distance,io_depth}', prosrc => 'pg_stat_get_recovery_prefetch' }, { oid => '2306', descr => 'statistics: information about SLRU caches', diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 38c9b04f72c..99ecf72970e 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -2238,11 +2238,12 @@ pg_stat_recovery_prefetch| SELECT stats_reset, skip_init, skip_new, skip_fpw, + skip_fpi, skip_rep, wal_distance, block_distance, io_depth - FROM pg_stat_get_recovery_prefetch() s(stats_reset, prefetch, hit, skip_init, skip_new, skip_fpw, skip_rep, wal_distance, block_distance, io_depth); + FROM pg_stat_get_recovery_prefetch() s(stats_reset, prefetch, hit, skip_init, skip_new, skip_fpw, skip_fpi, skip_rep, wal_distance, block_distance, io_depth); pg_stat_replication| SELECT s.pid, s.usesysid, u.rolname AS usename, -- 2.55.0
From 408d1f73a60949049118af726cd2a2c496d0dc4b Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 12:36:39 +0100 Subject: [PATCH v1 3/9] Rework the FPI cache as a set-associative cache The cache of recent FPIs was implemented as a simple array with a linear search, but that does not scale to longer histories. Replace it with a set-associative cache inspired by the fast-path lock approach. The array (with 256 entries) is divided into 16-entry sets, and each block is mapped to one of the sets. There's no explicit eviction of entries, the code relies on it to happen implicitly when adding new FPI entries (which replaces the oldest entry, determined by LSN). The 256 entry history is somewhat arbitrary. We only really need it to be large enough to prevent prefetching the block before the record gets replayed (which loads the block into shared buffers). --- src/backend/access/transam/xlogprefetcher.c | 82 +++++++++++++++++---- 1 file changed, 66 insertions(+), 16 deletions(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index ed06c3e429f..621a974c685 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -62,10 +62,22 @@ * A small history of most recently seen FPIs, so that we can skip useless * prefetches for these blocks (most likely still in cache). * - * XXX At which point should we switch to a hash table, instead of a linear - * search in a small array? + * The history is a set-associative hash table, i.e. each block maps to a group + * of slots, and it can only be stored in one of the slots of that group. */ -#define XLOGPREFETCHER_FPI_HISTORY_SIZE 64 +#define XLOGPREFETCHER_FPI_GROUPS 16 +#define XLOGPREFETCHER_FPI_PER_GROUP 16 +#define XLOGPREFETCHER_FPI_HISTORY_SIZE \ + (XLOGPREFETCHER_FPI_GROUPS * XLOGPREFETCHER_FPI_PER_GROUP) + +/* + * Trivial hash function mapping block to a group in the hash table, using + * small prime numbers. + */ +#define XLOGPREFETCHER_FPI_HASH(blkno, rlocator) \ + ((blkno) + 31 * (rlocator).spcOid + 83 * (rlocator).dbOid + 127 * (rlocator).relNumber) +#define XLOGPREFETCHER_FPI_GROUP(blkno, rlocator) \ + (XLOGPREFETCHER_FPI_HASH((blkno), (rlocator)) % XLOGPREFETCHER_FPI_GROUPS) /* * When maintenance_io_concurrency is not saturated, we're prepared to look @@ -155,7 +167,7 @@ struct XLogPrefetcher /* Book-keeping to avoid prefetching right after FPI */ RelFileLocator recent_fpi_rlocator[XLOGPREFETCHER_FPI_HISTORY_SIZE]; BlockNumber recent_fpi_block[XLOGPREFETCHER_FPI_HISTORY_SIZE]; - int recent_fpi_idx; + XLogRecPtr recent_fpi_lsn[XLOGPREFETCHER_FPI_HISTORY_SIZE]; /* Book-keeping to disable prefetching temporarily. */ XLogRecPtr no_readahead_until; @@ -704,11 +716,44 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) * duplicate FPI, it means the system is almost idle. We also * don't bother with evicting blocks from the history, for the * same reason. + * + * XXX Could we see multiple FPIs for the same block within + * the same checkpoint interval? E.g. during a backup, etc? + * Then maybe we should try to deduplicate the entries a bit + * more aggressively. */ - prefetcher->recent_fpi_rlocator[prefetcher->recent_fpi_idx] = block->rlocator; - prefetcher->recent_fpi_block[prefetcher->recent_fpi_idx] = block->blkno; - prefetcher->recent_fpi_idx = - (prefetcher->recent_fpi_idx + 1) % XLOGPREFETCHER_FPI_HISTORY_SIZE; + + /* add the FPI to the oldest (or empty) slot in the group */ + { + int group = XLOGPREFETCHER_FPI_GROUP(block->blkno, block->rlocator); + int slot = (group * XLOGPREFETCHER_FPI_PER_GROUP); /* first slot */ + + for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) + { + /* index in the whole arrays */ + int idx = (group * XLOGPREFETCHER_FPI_PER_GROUP + i); + + /* + * found an empty slot, so just use that + * + * XXX We could also have a cutoff age, and consider + * older slots as empty. + */ + if (prefetcher->recent_fpi_lsn[idx] == InvalidXLogRecPtr) + { + slot = idx; + break; + } + + /* use the older of the two slots */ + if (prefetcher->recent_fpi_lsn[idx] < prefetcher->recent_fpi_lsn[slot]) + slot = idx; + } + + prefetcher->recent_fpi_rlocator[slot] = block->rlocator; + prefetcher->recent_fpi_block[slot] = block->blkno; + prefetcher->recent_fpi_lsn[slot] = record->lsn; + } return LRQ_NEXT_NO_IO; } @@ -748,17 +793,22 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) * it from an FPI. We keep a bit longer history of FPIs. * * XXX Maybe we could skip this with full-page writes disabled? - * - * XXX Maybe we could skip this backwards, with the assumption - * that if there is a FPI, it's likely very close? */ - for (int i = 0; i < XLOGPREFETCHER_FPI_HISTORY_SIZE; ++i) { - if (block->blkno == prefetcher->recent_fpi_block[i] && - RelFileLocatorEquals(block->rlocator, prefetcher->recent_fpi_rlocator[i])) + int group = XLOGPREFETCHER_FPI_GROUP(block->blkno, block->rlocator); + + /* scan the whole group */ + for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; ++i) { - XLogPrefetchIncrement(&SharedStats->skip_fpi); - return LRQ_NEXT_NO_IO; + int idx = (group * XLOGPREFETCHER_FPI_PER_GROUP + i); + + if (prefetcher->recent_fpi_lsn[idx] != InvalidXLogRecPtr && + block->blkno == prefetcher->recent_fpi_block[idx] && + RelFileLocatorEquals(block->rlocator, prefetcher->recent_fpi_rlocator[idx])) + { + XLogPrefetchIncrement(&SharedStats->skip_fpi); + return LRQ_NEXT_NO_IO; + } } } -- 2.55.0
From 3b50cda951d93f41e21542909a541dda20bd6d8e Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 17:20:39 +0100 Subject: [PATCH v1 4/9] Move FPI tracking into separate functions Replace the anonymous blocks with static inline functions, to make the code more readable. While at it, store the history as a single array of structs instead of three parallel arrays, so that all fields of an entry are next to each other (and usually on the same cache line). --- src/backend/access/transam/xlogprefetcher.c | 166 ++++++++++++-------- src/tools/pgindent/typedefs.list | 1 + 2 files changed, 104 insertions(+), 63 deletions(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 621a974c685..86478c01708 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -141,6 +141,17 @@ typedef struct LsnReadQueue } queue[FLEXIBLE_ARRAY_MEMBER]; } LsnReadQueue; +/* + * An entry in the history of recently seen FPIs. Only main fork blocks are + * tracked, so there's no need to remember the fork number. + */ +typedef struct XLogPrefetcherFPI +{ + RelFileLocator rlocator; + BlockNumber blkno; + XLogRecPtr lsn; /* InvalidXLogRecPtr means the slot is empty */ +} XLogPrefetcherFPI; + /* * A prefetcher. This is a mechanism that wraps an XLogReader, prefetching * blocks that will be soon be referenced, to try to avoid IO stalls. @@ -165,9 +176,7 @@ struct XLogPrefetcher int recent_idx; /* Book-keeping to avoid prefetching right after FPI */ - RelFileLocator recent_fpi_rlocator[XLOGPREFETCHER_FPI_HISTORY_SIZE]; - BlockNumber recent_fpi_block[XLOGPREFETCHER_FPI_HISTORY_SIZE]; - XLogRecPtr recent_fpi_lsn[XLOGPREFETCHER_FPI_HISTORY_SIZE]; + XLogPrefetcherFPI recent_fpi[XLOGPREFETCHER_FPI_HISTORY_SIZE]; /* Book-keeping to disable prefetching temporarily. */ XLogRecPtr no_readahead_until; @@ -223,6 +232,13 @@ static inline bool XLogPrefetcherIsFiltered(XLogPrefetcher *prefetcher, BlockNumber blockno); static inline void XLogPrefetcherCompleteFilters(XLogPrefetcher *prefetcher, XLogRecPtr replaying_lsn); +static inline void XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, + RelFileLocator rlocator, + BlockNumber blkno, + XLogRecPtr lsn); +static inline bool XLogPrefetcherHasFPI(XLogPrefetcher *prefetcher, + RelFileLocator rlocator, + BlockNumber blkno); static LsnReadQueueNextStatus XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn); @@ -709,51 +725,9 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) { XLogPrefetchIncrement(&SharedStats->skip_fpw); - /* - * We should not really see a FPI for the same block very - * often, not more than once per checkpoint cycle. So don't - * bother with scanning the history here. If we end up with a - * duplicate FPI, it means the system is almost idle. We also - * don't bother with evicting blocks from the history, for the - * same reason. - * - * XXX Could we see multiple FPIs for the same block within - * the same checkpoint interval? E.g. during a backup, etc? - * Then maybe we should try to deduplicate the entries a bit - * more aggressively. - */ - - /* add the FPI to the oldest (or empty) slot in the group */ - { - int group = XLOGPREFETCHER_FPI_GROUP(block->blkno, block->rlocator); - int slot = (group * XLOGPREFETCHER_FPI_PER_GROUP); /* first slot */ - - for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) - { - /* index in the whole arrays */ - int idx = (group * XLOGPREFETCHER_FPI_PER_GROUP + i); - - /* - * found an empty slot, so just use that - * - * XXX We could also have a cutoff age, and consider - * older slots as empty. - */ - if (prefetcher->recent_fpi_lsn[idx] == InvalidXLogRecPtr) - { - slot = idx; - break; - } - - /* use the older of the two slots */ - if (prefetcher->recent_fpi_lsn[idx] < prefetcher->recent_fpi_lsn[slot]) - slot = idx; - } - - prefetcher->recent_fpi_rlocator[slot] = block->rlocator; - prefetcher->recent_fpi_block[slot] = block->blkno; - prefetcher->recent_fpi_lsn[slot] = record->lsn; - } + /* Remember the FPI, so that we can skip later references. */ + XLogPrefetcherAddFPI(prefetcher, block->rlocator, block->blkno, + record->lsn); return LRQ_NEXT_NO_IO; } @@ -794,22 +768,10 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) * * XXX Maybe we could skip this with full-page writes disabled? */ + if (XLogPrefetcherHasFPI(prefetcher, block->rlocator, block->blkno)) { - int group = XLOGPREFETCHER_FPI_GROUP(block->blkno, block->rlocator); - - /* scan the whole group */ - for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; ++i) - { - int idx = (group * XLOGPREFETCHER_FPI_PER_GROUP + i); - - if (prefetcher->recent_fpi_lsn[idx] != InvalidXLogRecPtr && - block->blkno == prefetcher->recent_fpi_block[idx] && - RelFileLocatorEquals(block->rlocator, prefetcher->recent_fpi_rlocator[idx])) - { - XLogPrefetchIncrement(&SharedStats->skip_fpi); - return LRQ_NEXT_NO_IO; - } - } + XLogPrefetchIncrement(&SharedStats->skip_fpi); + return LRQ_NEXT_NO_IO; } /* We're going to prefetch the block, so add it to history. */ @@ -1064,6 +1026,84 @@ XLogPrefetcherIsFiltered(XLogPrefetcher *prefetcher, RelFileLocator rlocator, return false; } +/* + * Return the group of slots in the history of recent FPIs, that the given + * block maps to. + */ +static inline XLogPrefetcherFPI * +XLogPrefetcherFPIGroup(XLogPrefetcher *prefetcher, RelFileLocator rlocator, + BlockNumber blkno) +{ + int group = XLOGPREFETCHER_FPI_GROUP(blkno, rlocator); + + return &prefetcher->recent_fpi[group * XLOGPREFETCHER_FPI_PER_GROUP]; +} + +/* + * Remember that the given block is restored from an FPI in the WAL record + * with the given LSN. + * + * We should not really see a FPI for the same block very often, not more + * than once per checkpoint cycle. So don't bother checking whether the + * block already is in the history. If we end up with a duplicate FPI, it + * means the system is almost idle. We also don't bother with evicting + * blocks from the history, for the same reason. + * + * XXX Could we see multiple FPIs for the same block within the same + * checkpoint interval? E.g. during a backup, etc? Then maybe we should + * try to deduplicate the entries a bit more aggressively. + */ +static inline void +XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, + BlockNumber blkno, XLogRecPtr lsn) +{ + XLogPrefetcherFPI *slots = XLogPrefetcherFPIGroup(prefetcher, rlocator, + blkno); + XLogPrefetcherFPI *slot = &slots[0]; + + /* + * Add the FPI to the first empty slot in the group, or to the oldest one. + * + * XXX We could also have a cutoff age, and consider older slots as empty. + */ + for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) + { + if (slots[i].lsn == InvalidXLogRecPtr) + { + slot = &slots[i]; + break; + } + + if (slots[i].lsn < slot->lsn) + slot = &slots[i]; + } + + slot->rlocator = rlocator; + slot->blkno = blkno; + slot->lsn = lsn; +} + +/* + * Check if the given block is in the history of recent FPIs. + */ +static inline bool +XLogPrefetcherHasFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, + BlockNumber blkno) +{ + XLogPrefetcherFPI *slots = XLogPrefetcherFPIGroup(prefetcher, rlocator, + blkno); + + for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) + { + if (slots[i].lsn != InvalidXLogRecPtr && + slots[i].blkno == blkno && + RelFileLocatorEquals(slots[i].rlocator, rlocator)) + return true; + } + + return false; +} + /* * A wrapper for XLogBeginRead() that also resets the prefetcher. */ diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index a7a8dbdd23f..a90cce6eb70 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -3499,6 +3499,7 @@ XLogPageReadPrivate XLogPageReadResult XLogPrefetchStats XLogPrefetcher +XLogPrefetcherFPI XLogPrefetcherFilter XLogReaderRoutine XLogReaderState -- 2.55.0
From 84ec75dd4c81dc51db11b1a093deae6b5330036d Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 18:10:10 +0100 Subject: [PATCH v1 5/9] Use a proper hash function for FPI history The history of recent FPIs is a set-associative table, with each block mapped to one of 16 groups using a trivial hash function (blkno + 31 * spcOid + 83 * dbOid + 127 * relNumber) % 16 The number of groups is a power of two, so only the lowest 4 bits of each input affect the result. And because 31 and 127 are -1 and 83 is 3 modulo 16, the group is (blkno - spcOid + 3 * dbOid - relNumber) % 16. For blocks of a given relation, the group is determined simply by the lowest 4 bits of the block number. That's not a particularly great hash function, obviously. If blocks are modified with a power-of-2 stride, only some of the groups would get used. With a stride of 16, all blocks map to the same group, and only the last 16 FPIs would be remembered. We could probably fix this by picking different coefficients in the formula, and/or maybe combine the values in a different way. But it's better (and ultimately cheaper, considering the consequences of poor caching) to use a proven hash function, combined with hash_combine(). --- src/backend/access/transam/xlogprefetcher.c | 25 ++++++++++++--------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 86478c01708..8d8d1a1ed2b 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -32,6 +32,7 @@ #include "catalog/pg_control.h" #include "catalog/storage_xlog.h" #include "commands/dbcommands_xlog.h" +#include "common/hashfn.h" #include "funcapi.h" #include "miscadmin.h" #include "port/atomics.h" @@ -70,15 +71,6 @@ #define XLOGPREFETCHER_FPI_HISTORY_SIZE \ (XLOGPREFETCHER_FPI_GROUPS * XLOGPREFETCHER_FPI_PER_GROUP) -/* - * Trivial hash function mapping block to a group in the hash table, using - * small prime numbers. - */ -#define XLOGPREFETCHER_FPI_HASH(blkno, rlocator) \ - ((blkno) + 31 * (rlocator).spcOid + 83 * (rlocator).dbOid + 127 * (rlocator).relNumber) -#define XLOGPREFETCHER_FPI_GROUP(blkno, rlocator) \ - (XLOGPREFETCHER_FPI_HASH((blkno), (rlocator)) % XLOGPREFETCHER_FPI_GROUPS) - /* * When maintenance_io_concurrency is not saturated, we're prepared to look * ahead up to N times that number of block references. @@ -1029,12 +1021,25 @@ XLogPrefetcherIsFiltered(XLogPrefetcher *prefetcher, RelFileLocator rlocator, /* * Return the group of slots in the history of recent FPIs, that the given * block maps to. + * + * The hash has to depend on all bits of the block number and the relation, + * otherwise blocks modified with a regular stride (or blocks of relations + * with some relfilenumbers) would map to only some of the groups, and the + * history would forget FPIs while other groups have plenty of space. */ static inline XLogPrefetcherFPI * XLogPrefetcherFPIGroup(XLogPrefetcher *prefetcher, RelFileLocator rlocator, BlockNumber blkno) { - int group = XLOGPREFETCHER_FPI_GROUP(blkno, rlocator); + uint32 hash; + uint32 group; + + hash = murmurhash32(blkno); + hash = hash_combine(hash, murmurhash32(rlocator.relNumber)); + hash = hash_combine(hash, murmurhash32(rlocator.dbOid)); + hash = hash_combine(hash, murmurhash32(rlocator.spcOid)); + + group = hash % XLOGPREFETCHER_FPI_GROUPS; return &prefetcher->recent_fpi[group * XLOGPREFETCHER_FPI_PER_GROUP]; } -- 2.55.0
From ab7e3e8d65b38811374d2cca0c7edbda30558582 Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 17:35:10 +0100 Subject: [PATCH v1 6/9] Ignore replayed FPIs in prefetch FPI history The recent FPI entries were not explicitly evicted from the history, with the justification that in an idle system it does not matter (because the blocks are likely still in memory), and on a busy system it's likely to get evicted due to recovery processing other FPIs. But we can actually do better - we only really care about prefetches before the WAL record with the FPI gets replayed - after that, we can process prefetches as usual (and it'll be skipped if the block is still in shared buffers). Which means we can evict entries from the history simply by ignoring those with LSN before replaying_lsn. If the buffer is still in shared buffers, this has the benefit that PrefetchSharedBuffer() may find it and pass it to redo as recent_buffer, saving a buffer mapping lookup. On idle systems this likely doesn't matter much, and on busy systems (doing a lot of writes with FPIs) the entries would get quickly evicted by incoming FPIs anyway. But it can also happen that the blocks are evicted by queries on the replica, or even by non-FPI recovery writes. But in such cases the extra I/O (due to synchronous reads) is likely negligible. Anyway, it's trivial to simply ignore entries for already replayed FPIs. The expired entries are effectively empty, and because they're older than any entry that still matters, the existing eviction policy (replace the oldest entry in the group) reuses them first. The history is reset whenever the prefetcher gets reconfigured, which happens after XLogPrefetcherBeginRead(), after a timeline switch, and whenever recovery_prefetch or maintenance_io_concurrency is set (e.g. on a configuration reload). XLogPrefetcherBeginRead() may move the reader to a different position, and FPIs read ahead before that may never get replayed. Their entries would suppress prefetches of those blocks until replay gets past their LSN (if ever). Currently, that reset is merely a precaution. XLogPrefetcherBeginRead() is called before redo starts, when nothing has been read ahead yet, and in FinishWalRecovery(). Only the latter can find pending entries (after stopping at a recovery target), and nothing gets replayed after it. Nothing is pending at a timeline switch either, because read-ahead stops at records that may switch the timeline. Setting the GUCs during redo may forget FPIs that are still pending, but that only means a few useless prefetches, as without the history. --- src/backend/access/transam/xlogprefetcher.c | 51 +++++++++++++++------ 1 file changed, 37 insertions(+), 14 deletions(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 8d8d1a1ed2b..3c5f7644491 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -60,8 +60,9 @@ #define XLOGPREFETCHER_SEQ_WINDOW_SIZE 4 /* - * A small history of most recently seen FPIs, so that we can skip useless - * prefetches for these blocks (most likely still in cache). + * A small history of most recently seen full-page images (FPIs), so that we + * can skip useless prefetches of blocks that are going to be restored from an + * FPI in a record that has not been replayed yet. * * The history is a set-associative hash table, i.e. each block maps to a group * of slots, and it can only be stored in one of the slots of that group. @@ -167,7 +168,7 @@ struct XLogPrefetcher BlockNumber recent_block[XLOGPREFETCHER_SEQ_WINDOW_SIZE]; int recent_idx; - /* Book-keeping to avoid prefetching right after FPI */ + /* Book-keeping to avoid prefetching blocks restored from FPIs. */ XLogPrefetcherFPI recent_fpi[XLOGPREFETCHER_FPI_HISTORY_SIZE]; /* Book-keeping to disable prefetching temporarily. */ @@ -230,7 +231,8 @@ static inline void XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, XLogRecPtr lsn); static inline bool XLogPrefetcherHasFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, - BlockNumber blkno); + BlockNumber blkno, + XLogRecPtr replaying_lsn); static LsnReadQueueNextStatus XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn); @@ -755,12 +757,15 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) } /* - * Likewise, no point in prefetching a block right after restoring - * it from an FPI. We keep a bit longer history of FPIs. + * Likewise, no point in prefetching a block that is going to be + * restored from an FPI in a record that has not been replayed + * yet. The block will be in shared buffers by the time this + * record gets replayed. * * XXX Maybe we could skip this with full-page writes disabled? */ - if (XLogPrefetcherHasFPI(prefetcher, block->rlocator, block->blkno)) + if (XLogPrefetcherHasFPI(prefetcher, block->rlocator, block->blkno, + replaying_lsn)) { XLogPrefetchIncrement(&SharedStats->skip_fpi); return LRQ_NEXT_NO_IO; @@ -1051,8 +1056,7 @@ XLogPrefetcherFPIGroup(XLogPrefetcher *prefetcher, RelFileLocator rlocator, * We should not really see a FPI for the same block very often, not more * than once per checkpoint cycle. So don't bother checking whether the * block already is in the history. If we end up with a duplicate FPI, it - * means the system is almost idle. We also don't bother with evicting - * blocks from the history, for the same reason. + * means the system is almost idle. * * XXX Could we see multiple FPIs for the same block within the same * checkpoint interval? E.g. during a backup, etc? Then maybe we should @@ -1069,7 +1073,9 @@ XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, /* * Add the FPI to the first empty slot in the group, or to the oldest one. * - * XXX We could also have a cutoff age, and consider older slots as empty. + * Entries for FPIs that have been replayed already are ignored by + * XLogPrefetcherHasFPI(), so they are effectively empty. They're also + * older than any entry that still matters, so they get reused first. */ for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) { @@ -1089,18 +1095,26 @@ XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, } /* - * Check if the given block is in the history of recent FPIs. + * Check if the given block is going to be restored from an FPI, i.e. if it's + * in the history of recent FPIs, and the record with the FPI has not been + * replayed yet. + * + * Once the record with the FPI is replayed, the block is in shared buffers, + * unless it got evicted since then. If it's still there, we want + * PrefetchSharedBuffer() to find it, so that XLogReadBufferForRedo() can use + * the buffer without another lookup. If it was evicted, it has to be read + * again, and prefetching it is useful. */ static inline bool XLogPrefetcherHasFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, - BlockNumber blkno) + BlockNumber blkno, XLogRecPtr replaying_lsn) { XLogPrefetcherFPI *slots = XLogPrefetcherFPIGroup(prefetcher, rlocator, blkno); for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) { - if (slots[i].lsn != InvalidXLogRecPtr && + if (slots[i].lsn > replaying_lsn && slots[i].blkno == blkno && RelFileLocatorEquals(slots[i].rlocator, rlocator)) return true; @@ -1115,7 +1129,7 @@ XLogPrefetcherHasFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, void XLogPrefetcherBeginRead(XLogPrefetcher *prefetcher, XLogRecPtr recPtr) { - /* This will forget about any in-flight IO. */ + /* This will forget about any in-flight IO, and about FPIs seen so far. */ prefetcher->reconfigure_count--; /* Book-keeping to avoid readahead on first read. */ @@ -1166,6 +1180,15 @@ XLogPrefetcherReadRecord(XLogPrefetcher *prefetcher, char **errmsg) (uintptr_t) prefetcher, XLogPrefetcherNextBlock); + /* + * Forget about FPIs seen so far. XLogPrefetcherBeginRead() may have + * moved the reader to a different position, and the entries would not + * be considered replayed until we replay past them again (if ever), + * so we might skip prefetching blocks that are not going to be + * restored from an FPI. + */ + memset(prefetcher->recent_fpi, 0, sizeof(prefetcher->recent_fpi)); + prefetcher->reconfigure_count = XLogPrefetchReconfigureCount; } -- 2.55.0
From cb1793f0f422ea20cf766c94eb39ba04db23d9fa Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 18:00:07 +0100 Subject: [PATCH v1 7/9] Avoid duplicate entries in FPI history Multiple FPIs for the same block within the look-ahead distance are rare, but can happen e.g. when a checkpoint starts between modifications of a block. Each duplicate wastes a slot, and may force out an entry for a different block that's still needed. When adding an FPI to the history, update the existing entry for the block, if there is one. XLogPrefetcherAddFPI() already scans the group when looking for the oldest entry, so this does not make it any more expensive. The empty slots are always at the end anyway. This also means XLogPrefetcherHasFPI() can be a bit smarter: * It can stop at the first (and only) entry for the block. Previously it kept looking after finding an entry for an already-replayed FPI. * It can stop at the first empty slot. Those are always at the end of a group, because new entries are added to the first empty slot, and slots only become empty when the whole history is reset. (In this context "empty" means an entry with invalid LSN.) --- src/backend/access/transam/xlogprefetcher.c | 33 +++++++++++++-------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 3c5f7644491..1f77e20c08b 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -1053,14 +1053,11 @@ XLogPrefetcherFPIGroup(XLogPrefetcher *prefetcher, RelFileLocator rlocator, * Remember that the given block is restored from an FPI in the WAL record * with the given LSN. * - * We should not really see a FPI for the same block very often, not more - * than once per checkpoint cycle. So don't bother checking whether the - * block already is in the history. If we end up with a duplicate FPI, it - * means the system is almost idle. - * - * XXX Could we see multiple FPIs for the same block within the same - * checkpoint interval? E.g. during a backup, etc? Then maybe we should - * try to deduplicate the entries a bit more aggressively. + * If the block already is in the history, the existing entry is updated, so + * that each block occupies at most one slot. It's not common to see multiple + * FPIs for the same block within the look-ahead distance, but it can happen, + * e.g. when a checkpoint starts between two modifications of the block, or + * with wal_consistency_checking (which adds an image to block references). */ static inline void XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, @@ -1071,7 +1068,11 @@ XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, XLogPrefetcherFPI *slot = &slots[0]; /* - * Add the FPI to the first empty slot in the group, or to the oldest one. + * Use the slot with the same block, or the first empty slot, or the + * oldest slot in the group. Empty slots are always at the end of the + * group, because we use the first empty slot, and slots only become empty + * when the whole history is reset. So if there is an entry for the same + * block, we find it before the first empty slot. * * Entries for FPIs that have been replayed already are ignored by * XLogPrefetcherHasFPI(), so they are effectively empty. They're also @@ -1079,7 +1080,9 @@ XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, */ for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) { - if (slots[i].lsn == InvalidXLogRecPtr) + if (slots[i].lsn == InvalidXLogRecPtr || + (slots[i].blkno == blkno && + RelFileLocatorEquals(slots[i].rlocator, rlocator))) { slot = &slots[i]; break; @@ -1114,10 +1117,14 @@ XLogPrefetcherHasFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) { - if (slots[i].lsn > replaying_lsn && - slots[i].blkno == blkno && + /* Empty slots are always at the end of the group. */ + if (slots[i].lsn == InvalidXLogRecPtr) + break; + + /* There's at most one entry for each block. */ + if (slots[i].blkno == blkno && RelFileLocatorEquals(slots[i].rlocator, rlocator)) - return true; + return (slots[i].lsn > replaying_lsn); } return false; -- 2.55.0
From 5bfdf0a2fa7ef416f3c893c7197cd230afac4ef0 Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 18:45:50 +0100 Subject: [PATCH v1 8/9] Size recovery FPI history by the look-ahead distance The history of recent FPIs had a fixed size of 256 slots (16 groups of 16 slots). But we only really care about FPIs that were not replayed yet, which is limited by the look-ahead distance. That is determined by maintenance_io_concurrency times XLOGPREFETCHER_DISTANCE_MULTIPLIER, i.e. 64 by default, up to 4000. When there are more FPIs in records that have not been replayed yet than the history can hold, useful entries may get evicted, and later references to those blocks are prefetched even though they are going to be restored from the FPI. If the blocks are modified in the same order as before, this may be painful, as we're evicting blocks in LSN order (classical LRU adversary case). There are never more than max_distance FPIs that still matter. We size the history based on that, with four slots per block reference within the look-ahead distance, rounded up to a power of two. This effectively oversizes the whole cache 4x, to make group overflows very rare. This is distinct from XLOGPREFETCHER_DISTANCE_MULTIPLIER, which is used to calculate look-ahead distance from maintenance_io_concurrency. The XLOGPREFETCHER_FPI_HISTORY_MULTIPLIER sizes the cache based on the prefetch distance. The history is allocated when the prefetcher is (re)configured, along with the LsnReadQueue, which also takes care of resetting it after XLogPrefetcherBeginRead(). As the number of groups is a power of two, the group can be determined using a mask instead of a division. --- src/backend/access/transam/xlogprefetcher.c | 43 +++++++++++++++------ 1 file changed, 32 insertions(+), 11 deletions(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index 1f77e20c08b..ba9e4e40a7e 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -36,6 +36,7 @@ #include "funcapi.h" #include "miscadmin.h" #include "port/atomics.h" +#include "port/pg_bitutils.h" #include "storage/bufmgr.h" #include "storage/fd.h" #include "storage/shmem.h" @@ -66,11 +67,16 @@ * * The history is a set-associative hash table, i.e. each block maps to a group * of slots, and it can only be stored in one of the slots of that group. + * + * Only FPIs in records that have not been replayed yet matter, and there are + * never more of those than the maximum look-ahead distance, because each of + * them occupies an entry in the LsnReadQueue until the record is replayed. + * The history has XLOGPREFETCHER_FPI_HISTORY_MULTIPLIER times as many slots + * (rounded up so that the number of groups is a power of two), so that a + * group overflows and forgets an FPI that still matters only very rarely. */ -#define XLOGPREFETCHER_FPI_GROUPS 16 #define XLOGPREFETCHER_FPI_PER_GROUP 16 -#define XLOGPREFETCHER_FPI_HISTORY_SIZE \ - (XLOGPREFETCHER_FPI_GROUPS * XLOGPREFETCHER_FPI_PER_GROUP) +#define XLOGPREFETCHER_FPI_HISTORY_MULTIPLIER 4 /* * When maintenance_io_concurrency is not saturated, we're prepared to look @@ -169,7 +175,8 @@ struct XLogPrefetcher int recent_idx; /* Book-keeping to avoid prefetching blocks restored from FPIs. */ - XLogPrefetcherFPI recent_fpi[XLOGPREFETCHER_FPI_HISTORY_SIZE]; + XLogPrefetcherFPI *recent_fpi; + uint32 recent_fpi_groups; /* always a power of two */ /* Book-keeping to disable prefetching temporarily. */ XLogRecPtr no_readahead_until; @@ -435,6 +442,8 @@ XLogPrefetcherFree(XLogPrefetcher *prefetcher) { lrq_free(prefetcher->streaming_read); hash_destroy(prefetcher->filter_table); + if (prefetcher->recent_fpi) + pfree(prefetcher->recent_fpi); pfree(prefetcher); } @@ -1044,7 +1053,7 @@ XLogPrefetcherFPIGroup(XLogPrefetcher *prefetcher, RelFileLocator rlocator, hash = hash_combine(hash, murmurhash32(rlocator.dbOid)); hash = hash_combine(hash, murmurhash32(rlocator.spcOid)); - group = hash % XLOGPREFETCHER_FPI_GROUPS; + group = hash & (prefetcher->recent_fpi_groups - 1); return &prefetcher->recent_fpi[group * XLOGPREFETCHER_FPI_PER_GROUP]; } @@ -1188,13 +1197,25 @@ XLogPrefetcherReadRecord(XLogPrefetcher *prefetcher, char **errmsg) XLogPrefetcherNextBlock); /* - * Forget about FPIs seen so far. XLogPrefetcherBeginRead() may have - * moved the reader to a different position, and the entries would not - * be considered replayed until we replay past them again (if ever), - * so we might skip prefetching blocks that are not going to be - * restored from an FPI. + * Allocate a new history of FPIs, sized for the new look-ahead + * distance. This also forgets about FPIs seen so far, which is + * necessary because XLogPrefetcherBeginRead() may have moved the + * reader to a different position, and the entries would not be + * considered replayed until we replay past them again (if ever), so + * we might skip prefetching blocks that are not going to be restored + * from an FPI. */ - memset(prefetcher->recent_fpi, 0, sizeof(prefetcher->recent_fpi)); + if (prefetcher->recent_fpi) + pfree(prefetcher->recent_fpi); + + prefetcher->recent_fpi_groups = + pg_nextpower2_32(Max(1, max_distance * + XLOGPREFETCHER_FPI_HISTORY_MULTIPLIER / + XLOGPREFETCHER_FPI_PER_GROUP)); + prefetcher->recent_fpi = + palloc0_array(XLogPrefetcherFPI, + prefetcher->recent_fpi_groups * + XLOGPREFETCHER_FPI_PER_GROUP); prefetcher->reconfigure_count = XLogPrefetchReconfigureCount; } -- 2.55.0
From 99047de70afa74c508b55f23ff47b2510117d550 Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Sun, 25 Oct 2026 19:11:35 +0100 Subject: [PATCH v1 9/9] Skip FPI lookups when no FPIs are pending XLogPrefetcherHasFPI() is called for every main fork block reference that gets past the other checks in XLogPrefetcherNextBlock(), i.e. for nearly every block that ends up being prefetched or found in shared buffers. It has to hash the block, and compare it with the entries in the group it maps to. But the entries only matter until the record with the FPI is replayed, and quite often there are no outstanding FPIs at all, e.g. later in a checkpoint cycle, when most of the modified blocks were already modified since the checkpoint. Track the highest LSN of all FPIs in the history, and return right away if the record with that LSN has been replayed already. This does not change which blocks get prefetched, it only skips searching the history when it can't find a pending FPI. --- src/backend/access/transam/xlogprefetcher.c | 26 +++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c index ba9e4e40a7e..1eddaac558e 100644 --- a/src/backend/access/transam/xlogprefetcher.c +++ b/src/backend/access/transam/xlogprefetcher.c @@ -177,6 +177,7 @@ struct XLogPrefetcher /* Book-keeping to avoid prefetching blocks restored from FPIs. */ XLogPrefetcherFPI *recent_fpi; uint32 recent_fpi_groups; /* always a power of two */ + XLogRecPtr recent_fpi_max_lsn; /* highest LSN in recent_fpi */ /* Book-keeping to disable prefetching temporarily. */ XLogRecPtr no_readahead_until; @@ -770,8 +771,6 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn) * restored from an FPI in a record that has not been replayed * yet. The block will be in shared buffers by the time this * record gets replayed. - * - * XXX Maybe we could skip this with full-page writes disabled? */ if (XLogPrefetcherHasFPI(prefetcher, block->rlocator, block->blkno, replaying_lsn)) @@ -1104,6 +1103,13 @@ XLogPrefetcherAddFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, slot->rlocator = rlocator; slot->blkno = blkno; slot->lsn = lsn; + + /* + * FPIs are normally added in LSN order, but use Max() so that the quick + * exit in XLogPrefetcherHasFPI() can't give a different answer than + * searching the history, no matter what. + */ + prefetcher->recent_fpi_max_lsn = Max(prefetcher->recent_fpi_max_lsn, lsn); } /* @@ -1121,8 +1127,19 @@ static inline bool XLogPrefetcherHasFPI(XLogPrefetcher *prefetcher, RelFileLocator rlocator, BlockNumber blkno, XLogRecPtr replaying_lsn) { - XLogPrefetcherFPI *slots = XLogPrefetcherFPIGroup(prefetcher, rlocator, - blkno); + XLogPrefetcherFPI *slots; + + /* + * Quick exit if all FPIs in the history have been replayed already. This + * is the common case when there are no FPIs within the look-ahead + * distance, e.g. with full_page_writes=off, or later in a checkpoint + * cycle, when most of the modified blocks were modified since the + * checkpoint already. + */ + if (prefetcher->recent_fpi_max_lsn <= replaying_lsn) + return false; + + slots = XLogPrefetcherFPIGroup(prefetcher, rlocator, blkno); for (int i = 0; i < XLOGPREFETCHER_FPI_PER_GROUP; i++) { @@ -1216,6 +1233,7 @@ XLogPrefetcherReadRecord(XLogPrefetcher *prefetcher, char **errmsg) palloc0_array(XLogPrefetcherFPI, prefetcher->recent_fpi_groups * XLOGPREFETCHER_FPI_PER_GROUP); + prefetcher->recent_fpi_max_lsn = InvalidXLogRecPtr; prefetcher->reconfigure_count = XLogPrefetchReconfigureCount; } -- 2.55.0
