Amit, Vignesh, Agreed, and thanks Vignesh for splitting it out. v13 is what I had in mind: the torn-page read fix on its own, backpatchable, with the recovery test and the HEAD-only rework (dropping the pin-only registration, removing REGBUF_NO_CHANGE) left for a separate commit on master once this is in.
I confirmed the two patches apply where Vignesh said: v13-0001 from master through PG17, and v13_REL16-0001 on PG16 and PG15. I am re-attaching Vignesh's two patches unchanged so this message has the reviewed set in one place. No change from his v13. For the follow-up on master I will post the test and the pin-only rework as their own thread-continuation once the fix lands, including the arch-independent torn-page test (it keys the deletion off the actual overflow-page contents with pageinspect, so it holds on 32-bit, and writes backup_label in binary mode so it holds on Windows). No need to hold the fix for that. -greg
From b9f57580de5eae940dd7acd1b439d20d5ce1e6fc Mon Sep 17 00:00:00 2001 From: Vignesh C <[email protected]> Date: Tue, 17 Nov 2026 09:10:56 +0530 Subject: [PATCH v13] Fix hash index redo reading a torn primary bucket page Replay of XLOG_HASH_DELETE, XLOG_HASH_MOVE_PAGE_CONTENTS and XLOG_HASH_SQUEEZE_PAGE reads the primary bucket page only to take a cleanup lock on it, so that no scan on a hot standby runs in the bucket while tuples are removed or moved. VACUUM holds only a pin on that page, so the record does not modify it and carries no full-page image of it; redo reads it from disk with RBM_NORMAL. If the page is torn on disk and the record that repairs it with a full-page image comes later in the WAL, recovery fails when it reaches one of these records before the image, because the RBM_NORMAL read of the torn page raises "invalid page". This happens both in crash recovery (the image is in WAL generated after the last checkpoint) and on a standby started from a base backup whose copy of the page is torn (the image is in WAL generated during the backup). Robert Haas noted this torn-page hazard during the original hash WAL work in 2017; redo was switched to XLogReadBufferForRedoExtended() so it restores an image when the record has one, but these records register the primary bucket page pin-only with REGBUF_NO_IMAGE, so there is no image to restore and the RBM_NORMAL read still reaches the torn page. The cleanup lock matters only to serialize against hot-standby scans, which can run only once the standby accepts queries, that is once recovery has reached consistency. Until then the read and lock are unnecessary, so skip them when HotStandbyActiveInReplay() is false. That also avoids reading the page in the window where its torn on-disk copy has not yet been repaired, since the repairing image always precedes the consistency point. nbtree and gist already gate redo cleanup-lock work on the hot-standby state for the same reason. Reported-by: shihao zhong <[email protected]> Author: Greg Burd <[email protected]> Co-authored-by: Vignesh C <[email protected]> Suggested-by: Amit Kapila <[email protected]> Reviewed-by: Amit Kapila <[email protected]> Reviewed-by: Joao Rito <[email protected]> Discussion: https://postgr.es/m/CAGRkXqT_AiwcsWZg5hnR1L2oWO8UpQpkL0iAjoZR+bey=go...@mail.gmail.com Discussion: https://postgr.es/m/caa4ek1lwzk0mak0lnczz6hsqchnamfjb47na8xuwrp4cylf...@mail.gmail.com Backpatch-through: 15 --- src/backend/access/hash/hash_xlog.c | 52 +++++++++++++++++++---- src/backend/access/transam/xlogrecovery.c | 3 +- src/include/access/xlogrecovery.h | 1 + 3 files changed, 45 insertions(+), 11 deletions(-) diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index e9a2b9aa9a7..46047c6ea27 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -17,6 +17,7 @@ #include "access/bufmask.h" #include "access/hash.h" #include "access/hash_xlog.h" +#include "access/xlogrecovery.h" #include "access/xlogutils.h" #include "storage/standby.h" @@ -494,10 +495,21 @@ hash_xlog_move_page_contents(XLogReaderState *record) else { /* - * we don't care for return value as the purpose of reading bucketbuf - * is to ensure a cleanup lock on primary bucket page. + * The cleanup lock is needed only when hot standby is active, because + * scans cannot run before that. Skipping the read also avoids a torn + * copy of the primary page. This record does not modify that page, so + * it has no full-page image of it. In crash recovery, or on a standby + * started from a base backup, the page on disk can be torn and the + * image that repairs it can come later in the WAL. */ - (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + if (HotStandbyActiveInReplay()) + { + /* + * we don't care for return value as the purpose of reading + * bucketbuf is to ensure a cleanup lock on primary bucket page. + */ + (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + } action = XLogReadBufferForRedo(record, 1, &writebuf); } @@ -621,10 +633,21 @@ hash_xlog_squeeze_page(XLogReaderState *record) else { /* - * we don't care for return value as the purpose of reading bucketbuf - * is to ensure a cleanup lock on primary bucket page. + * The cleanup lock is needed only when hot standby is active, because + * scans cannot run before that. Skipping the read also avoids a torn + * copy of the primary page. This record does not modify that page, so + * it has no full-page image of it. In crash recovery, or on a standby + * started from a base backup, the page on disk can be torn and the + * image that repairs it can come later in the WAL. */ - (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + if (HotStandbyActiveInReplay()) + { + /* + * we don't care for return value as the purpose of reading + * bucketbuf is to ensure a cleanup lock on primary bucket page. + */ + (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + } if (xldata->ntups > 0 || xldata->is_prev_bucket_same_wrt) action = XLogReadBufferForRedo(record, 1, &writebuf); @@ -852,10 +875,21 @@ hash_xlog_delete(XLogReaderState *record) else { /* - * we don't care for return value as the purpose of reading bucketbuf - * is to ensure a cleanup lock on primary bucket page. + * The cleanup lock is needed only when hot standby is active, because + * scans cannot run before that. Skipping the read also avoids a torn + * copy of the primary page. This record does not modify that page, so + * it has no full-page image of it. In crash recovery, or on a standby + * started from a base backup, the page on disk can be torn and the + * image that repairs it can come later in the WAL. */ - (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + if (HotStandbyActiveInReplay()) + { + /* + * we don't care for return value as the purpose of reading + * bucketbuf is to ensure a cleanup lock on primary bucket page. + */ + (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + } action = XLogReadBufferForRedo(record, 1, &deletebuf); } diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 52ac6d58423..e091a3e178b 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -390,7 +390,6 @@ static int XLogFileReadAnyTLI(XLogSegNo segno, XLogSource source); static bool CheckForStandbyTrigger(void); static void SetPromoteIsTriggered(void); -static bool HotStandbyActiveInReplay(void); static void SetCurrentChunkStartTime(TimestampTz xtime); static void SetLatestXTime(TimestampTz xtime); @@ -4589,7 +4588,7 @@ HotStandbyActive(void) * Like HotStandbyActive(), but to be used only in WAL replay code, * where we don't need to ask any other process what the state is. */ -static bool +bool HotStandbyActiveInReplay(void) { Assert(AmStartupProcess() || !IsPostmasterEnvironment); diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 8786b6d3a8e..11d3eadef3e 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -216,6 +216,7 @@ extern void ShutdownWalRecovery(void); extern void RemovePromoteSignalFiles(void); extern bool HotStandbyActive(void); +extern bool HotStandbyActiveInReplay(void); extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI); extern RecoveryPauseState GetRecoveryPauseState(void); extern void SetRecoveryPause(bool recoveryPause); -- 2.55.0
From 3ca6f01207769ab53524f4341a66f0cf3984a61d Mon Sep 17 00:00:00 2001 From: Vignesh C <[email protected]> Date: Tue, 17 Nov 2026 16:38:34 +0530 Subject: [PATCH v13_REL16] Fix hash index redo reading a torn primary bucket page Replay of XLOG_HASH_DELETE, XLOG_HASH_MOVE_PAGE_CONTENTS and XLOG_HASH_SQUEEZE_PAGE reads the primary bucket page only to take a cleanup lock on it, so that no scan on a hot standby runs in the bucket while tuples are removed or moved. VACUUM holds only a pin on that page, so the record does not modify it and carries no full-page image of it; redo reads it from disk with RBM_NORMAL. If the page is torn on disk and the record that repairs it with a full-page image comes later in the WAL, recovery fails when it reaches one of these records before the image, because the RBM_NORMAL read of the torn page raises "invalid page". This happens both in crash recovery (the image is in WAL generated after the last checkpoint) and on a standby started from a base backup whose copy of the page is torn (the image is in WAL generated during the backup). Robert Haas noted this torn-page hazard during the original hash WAL work in 2017; redo was switched to XLogReadBufferForRedoExtended() so it restores an image when the record has one, but these records register the primary bucket page pin-only with REGBUF_NO_IMAGE, so there is no image to restore and the RBM_NORMAL read still reaches the torn page. The cleanup lock matters only to serialize against hot-standby scans, which can run only once the standby accepts queries, that is once recovery has reached consistency. Until then the read and lock are unnecessary, so skip them when HotStandbyActiveInReplay() is false. That also avoids reading the page in the window where its torn on-disk copy has not yet been repaired, since the repairing image always precedes the consistency point. nbtree and gist already gate redo cleanup-lock work on the hot-standby state for the same reason. Reported-by: shihao zhong <[email protected]> Author: Greg Burd <[email protected]> Co-authored-by: Vignesh C <[email protected]> Suggested-by: Amit Kapila <[email protected]> Reviewed-by: Amit Kapila <[email protected]> Reviewed-by: Joao Rito <[email protected]> Discussion: https://postgr.es/m/CAGRkXqT_AiwcsWZg5hnR1L2oWO8UpQpkL0iAjoZR+bey=go...@mail.gmail.com Discussion: https://postgr.es/m/caa4ek1lwzk0mak0lnczz6hsqchnamfjb47na8xuwrp4cylf...@mail.gmail.com --- src/backend/access/hash/hash_xlog.c | 52 +++++++++++++++++++---- src/backend/access/transam/xlogrecovery.c | 3 +- src/include/access/xlogrecovery.h | 1 + 3 files changed, 45 insertions(+), 11 deletions(-) diff --git a/src/backend/access/hash/hash_xlog.c b/src/backend/access/hash/hash_xlog.c index cd9617533c2..f2edf15f2b4 100644 --- a/src/backend/access/hash/hash_xlog.c +++ b/src/backend/access/hash/hash_xlog.c @@ -19,6 +19,7 @@ #include "access/hash_xlog.h" #include "access/transam.h" #include "access/xlog.h" +#include "access/xlogrecovery.h" #include "access/xlogutils.h" #include "miscadmin.h" #include "storage/procarray.h" @@ -499,10 +500,21 @@ hash_xlog_move_page_contents(XLogReaderState *record) else { /* - * we don't care for return value as the purpose of reading bucketbuf - * is to ensure a cleanup lock on primary bucket page. + * The cleanup lock is needed only when hot standby is active, because + * scans cannot run before that. Skipping the read also avoids a torn + * copy of the primary page. This record does not modify that page, so + * it has no full-page image of it. In crash recovery, or on a standby + * started from a base backup, the page on disk can be torn and the + * image that repairs it can come later in the WAL. */ - (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + if (HotStandbyActiveInReplay()) + { + /* + * we don't care for return value as the purpose of reading + * bucketbuf is to ensure a cleanup lock on primary bucket page. + */ + (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + } action = XLogReadBufferForRedo(record, 1, &writebuf); } @@ -627,10 +639,21 @@ hash_xlog_squeeze_page(XLogReaderState *record) else { /* - * we don't care for return value as the purpose of reading bucketbuf - * is to ensure a cleanup lock on primary bucket page. + * The cleanup lock is needed only when hot standby is active, because + * scans cannot run before that. Skipping the read also avoids a torn + * copy of the primary page. This record does not modify that page, so + * it has no full-page image of it. In crash recovery, or on a standby + * started from a base backup, the page on disk can be torn and the + * image that repairs it can come later in the WAL. */ - (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + if (HotStandbyActiveInReplay()) + { + /* + * we don't care for return value as the purpose of reading + * bucketbuf is to ensure a cleanup lock on primary bucket page. + */ + (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + } action = XLogReadBufferForRedo(record, 1, &writebuf); } @@ -839,10 +862,21 @@ hash_xlog_delete(XLogReaderState *record) else { /* - * we don't care for return value as the purpose of reading bucketbuf - * is to ensure a cleanup lock on primary bucket page. + * The cleanup lock is needed only when hot standby is active, because + * scans cannot run before that. Skipping the read also avoids a torn + * copy of the primary page. This record does not modify that page, so + * it has no full-page image of it. In crash recovery, or on a standby + * started from a base backup, the page on disk can be torn and the + * image that repairs it can come later in the WAL. */ - (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + if (HotStandbyActiveInReplay()) + { + /* + * we don't care for return value as the purpose of reading + * bucketbuf is to ensure a cleanup lock on primary bucket page. + */ + (void) XLogReadBufferForRedoExtended(record, 0, RBM_NORMAL, true, &bucketbuf); + } action = XLogReadBufferForRedo(record, 1, &deletebuf); } diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 2bc615a691b..7f9ec6d8075 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -441,7 +441,6 @@ static int XLogFileReadAnyTLI(XLogSegNo segno, int emode, XLogSource source); static bool CheckForStandbyTrigger(void); static void SetPromoteIsTriggered(void); -static bool HotStandbyActiveInReplay(void); static void SetCurrentChunkStartTime(TimestampTz xtime); static void SetLatestXTime(TimestampTz xtime); @@ -4528,7 +4527,7 @@ HotStandbyActive(void) * Like HotStandbyActive(), but to be used only in WAL replay code, * where we don't need to ask any other process what the state is. */ -static bool +bool HotStandbyActiveInReplay(void) { Assert(AmStartupProcess() || !IsPostmasterEnvironment); diff --git a/src/include/access/xlogrecovery.h b/src/include/access/xlogrecovery.h index 7737f9e2cf0..0c25471c2e5 100644 --- a/src/include/access/xlogrecovery.h +++ b/src/include/access/xlogrecovery.h @@ -136,6 +136,7 @@ extern void ShutdownWalRecovery(void); extern void RemovePromoteSignalFiles(void); extern bool HotStandbyActive(void); +extern bool HotStandbyActiveInReplay(void); extern XLogRecPtr GetXLogReplayRecPtr(TimeLineID *replayTLI); extern RecoveryPauseState GetRecoveryPauseState(void); extern void SetRecoveryPause(bool recoveryPause); -- 2.55.0
