17.08.2026 11:29, Vadim Ponomarev пишет:
> Hi hackers,
>
> On a primary with synchronous replication, every commit that wrote WAL
> goes through SyncRepWaitForLSN(), and every standby reply goes through
> SyncRepReleaseWaiters(). Both take SyncRepLock exclusively, although
> much of their work does not need to happen under the lock.
>
> The attached series moves that work out of the critical section. The
> four patches are independent; only 0003 changes observable behaviour.
>
> 0001 -- Wake released waiters after dropping the queue lock.
>
> SyncRepWakeQueue() sets each released backend's latch while holding
> SyncRepLock. Setting a latch may call kill(), so at high commit rates
> the walsender makes one syscall per released commit while committers
> queue on the same lock.
>
> The patch collects released procs in a list and wakes them after dropping
> the lock, as ProcArrayGroupClearXid() already does with ProcArrayLock.
> The unlink, write barrier, and state update remain under the lock: a
> waiter reads syncRepState without it and must not see itself as completed
> while still on the queue.
>
> 0002 -- Compute synced positions before taking the queue lock.
>
> SyncRepReleaseWaiters() takes SyncRepLock before scanning the walsender
> slots for synced write, flush, and apply positions. That scan takes a
> spinlock per slot, allocates memory, and sorts quorum results.
>
> The patch moves the scan before the lock. The consumers only move lsn[]
> forward, so a result that becomes stale while waiting for the lock simply
> does not advance it. The patch also avoids the lock when the walsender
> is not a synchronous standby.
>
> 0003 -- Release waiters once per drained batch of standby replies.
>
> ProcessStandbyReplyMessage() calls SyncRepReleaseWaiters() for every
> reply, even when several replies are already waiting in the socket.
> Except for the last one, each pass then uses positions that the next
> message immediately replaces.
>
> The patch marks a release as pending and runs it once after draining the
> socket. The main risk is losing a pending release on an early exit: no
> other process will wake the committers acknowledged by that reply.
>
> The goodbye, EOF, invalid and unexpected message paths all run the
> pending release before returning. A normal standby shutdown sends its
> final reply and goodbye together, so that path matters in practice.
> Errors while parsing a later message, including a torn message, go
> through WalSndErrorCleanup(), which releases waiters after dropping the
> locks.
>
> 0004 -- Skip the lock when the acknowledgement has already arrived.
>
> This patch mirrors lsn[] in an atomic watermark and checks it before
> SyncRepWaitForLSN() takes the lock. The watermark is updated under the
> lock immediately after lsn[]. Both only move forward, so a stale read
> may take the slow path unnecessarily but cannot skip a required wait.
>
> I am least sure that 0004 is worth the extra shared state; see below.
>
> Prior work
> ----------
>
> 0001 was proposed in Thomas Munro's "Latches vs lwlock contention"
> thread as part of a general SetLatches() facility:
>
>
> https://www.postgresql.org/message-id/CA%2BhUKGKmO7ze0Z6WXKdrLxmvYa%3DzVGGXOO30MMktufofVwEm1A%40mail.gmail.com
>
> The heavyweight-lock part of that work was committed in November 2024,
> but SetLatches() and its sync-rep use were not. As far as I can tell,
> the sync-rep patch did not get a separate review. I used a list local to
> syncrep.c to keep the change contained and avoid the open questions
> around buffers and allocation in the general facility.
>
> There is also precedent for 0004. Michael Paquier's 2e57790836c ("Fix
> race with synchronous_standby_names at startup", April 2025) reads
> WalSndCtl->sync_standbys_status without the lock. The same monotonicity
> argument applies here; the LSN needs the atomics API because it is 64
> bits wide.
>
> Measurements
> ------------
>
> I compared devel master with the same master plus all four patches on
> two hosts connected by a dedicated 100 GbE link (RTT 0.11 ms):
>
> primary 4-socket Xeon Platinum 8580, 240 threads, 2 TB RAM, NVMe
> standby 2-socket Xeon Gold 5320, 104 threads, 1 TB RAM, NVMe
>
> The test used pgbench scale 2000, fillfactor 70, 750 clients,
> -M prepared, the built-in TPC-B script, and 10-minute runs with
> synchronous_commit = on, fsync and full_page_writes on. postgres used
> two primary sockets and pgbench a third.
>
> Each point started from the same prepared cluster, and the standby was
> rebuilt from a fresh base backup. I ran three interleaved pairs,
> alternating base and patched:
>
> tps runs 122250/122591/123091 -> 134849/134447/132740
> mean tps 122644 -> 134012 +9.3%
> mean latency 6.085 ms -> 5.558 ms -8.7%
> failed transactions: none
>
> The spread was 0.7% for base and 1.6% for patched.
>
> pg_stat_activity samples taken every 5 seconds show the same effect.
> The queue lock and standby acknowledgement wait are both named SyncRep;
> wait_event_type separates them:
>
> LWLock/SyncRep samples 12222 -> 6912 -43%
> IPC/SyncRep samples 20699 -> 14694 -29%
>
> Per committed transaction, lock-wait samples fell from 0.0997 to 0.0516.
> The queue-lock wait was roughly halved while throughput rose by 9%. The
> standby acknowledgement wait also fell by 35% per transaction, as
> committers stopped queueing for SyncRepLock before waiting for the
> standby.
>
> To see which patches contributed, I also ran -DLWLOCK_STATS builds on a
> small single-host setup: 16 threads, scale 20, 128 clients, 30 seconds.
> Across two runs, base -> patched:
>
> walsender acquisitions/commit 0.900-1.106 -> 0.538-0.610
> backend acquisitions/commit 0.9983-0.9995 -> 0.977-0.984
> commits that blocked 9.34-10.20% -> 0.24-0.27%
>
> At 32 clients, blocked acquisitions fell from 37809 to 362.
>
> 0003 makes most of the difference: the walsender takes the lock roughly
> half as often. 0004 rarely takes its fast path, saving only 1.6-2.3% of
> backend acquisitions at 128 clients. With synchronous_commit = on, it
> can only catch a commit if an acknowledgement for a later transaction
> happens to cover its LSN.
>
> I also ran four interleaved, 60-second single-client pairs. Mean
> latency was 0.760 ms on base and 0.762 ms patched, so deferring the
> release to the end of a one-reply drain showed no measurable delay.
>
> Testing
> -------
>
> 0003 adds src/test/recovery/t/056_syncrep_release.pl for two cases where
> a deferred release could be lost:
>
> * A stopped walsender resumes after the standby has applied the commit
> and shut down, then drains the final reply and goodbye together.
>
> * An injection point after a drained reply stands in for a torn message.
> The walreceiver is stopped while replay advances from WAL already on
> disk, so the first reply after resume carries the apply position the
> committer needs.
>
> Notes
> -------
>
> The SyncRepReleaseWaiters() call on configuration reload is outside the
> reply drain and is unchanged by 0003.
>
> 0001 allocates a MaxBackends-sized wake list once per process in
> TopMemoryContext. I am open to changing that if there is a better fit.
>
> The patches are against 7e6e294e4e4.
>
>
> -------
>
> Review would be especially helpful on two points:
>
> * Did I miss any exit path from the reply drain that must run the
> deferred release?
>
> * Is 0004 worth its new atomic in WalSndCtlData for a 1.6-2.3% reduction
> in backend lock acquisitions in these tests?
>
> I plan to register 0001-0003 for the September CommitFest and drop 0004,
> unless there is a workload where its fast path is more useful.
>
> Regards,
> Vadim Ponomarev
Good day, Vadim.
We've measured your patches and confirm they improve performance of
synchronous replication:
- with couple of 56 vcore virtual machines and pgbench running on replica,
250 clients, scale 2000, "TPC-B like" improved from 67.3kTPS to 70.4kTPS.
Which is quite impressive.
0001 i've found independently, so I fully share the idea. It really works.
0003 impressed me a lot. Great thing, imo!
0002 looks like "dirty-hack", but reading closely I found no issues:
- SyncRepGetCandidatesStandbys syncs by spinlocks on every walsender
- all WalSndCtl->lsn increases monotonically under lock.
0004 it really doesn't cost anything and gives some value. So let it be.
We didn't measure things one-by-one, but I suppose 0001 and 0003 gives most
of gain. Still other thing are useful as well, I believe 0002 and 0004 are
worth to be committed.
I've rebased patches and simplified a bit 0001 and 0004:
- 0001 already uses static variable. So why don't just make it file-wide
and use in SyncRepWakeQueue directly?
- 0004 - there is no need in separate lsn and lsn_published. Lets simply
convert lsn to atomic variable.
And I've refactored condition under lock a bit to make it clear why test
against of just atomic WalSndCtl->lsn[mode] could be enough. It was a bit
non-obvious in the form it is in master branch.
And I've added 027_stream_regress_sync.pl in 0005 as tweaked copy of
027_stream_regress.pl to test synchronous replication under concurrent load.
--
regards
Yura Sokolov aka funny-falcon
From 16b7ad82a521043f580bc1a1a9a60995f8ca3b57 Mon Sep 17 00:00:00 2001
From: Vadim Ponomarev <[email protected]>
Date: Sat, 15 Aug 2026 12:08:13 +0300
Subject: [PATCH v2 1/5] Wake the released sync-rep waiters after the queue
lock is down
SyncRepWakeQueue() sets each released backend's latch while holding
SyncRepLock exclusively. A latch is a kill() syscall whenever its proc is
asleep, and at a high commit rate the walsender runs one of them per
released commit inside the very section every committer lines up on.
ProcArrayGroupClearXid() already wakes its batch only after ProcArrayLock
is down, for the same reason.
Collect the released procs into a list instead, and set their latches once
the lock is released. The unlink, the write barrier and the state store
stay under the lock: a waiter reads syncRepState without the lock and must
never find itself completed while still on the queue. Nothing in the
deferred loop can error out, so a released proc cannot be left completed
but unlatched short of the process dying outright -- a window the in-lock
SetLatch had as well. A proc that noticed its state on its own and moved
on, even into a new wait, gets a spurious latch set, which every latch
sleeper tolerates.
The list is sized to MaxBackends and allocated once per releasing process.
A proc waits in at most one queue, so one list of that size bounds a walk
over all three.
---
src/backend/replication/syncrep.c | 72 ++++++++++++++++++++++++++++---
1 file changed, 65 insertions(+), 7 deletions(-)
diff --git a/src/backend/replication/syncrep.c b/src/backend/replication/syncrep.c
index d870f09e0a0..2824cc0e36f 100644
--- a/src/backend/replication/syncrep.c
+++ b/src/backend/replication/syncrep.c
@@ -84,6 +84,7 @@
#include "storage/proc.h"
#include "tcop/tcopprot.h"
#include "utils/guc_hooks.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/wait_event.h"
@@ -98,8 +99,16 @@ static bool announce_next_takeover = true;
SyncRepConfigData *SyncRepConfig = NULL;
static int SyncRepWaitMode = SYNC_REP_NO_WAIT;
+static struct
+{
+ Latch **arr;
+ int n;
+} SyncRepWakeList = {NULL, 0};
+
static void SyncRepQueueInsert(int mode);
static void SyncRepCancelWait(void);
+static void SyncRepInitWakeList(void);
+static void SyncRepWakeFromList(void);
static int SyncRepWakeQueue(bool all, int mode);
static bool SyncRepGetSyncRecPtr(XLogRecPtr *writePtr,
@@ -512,6 +521,8 @@ SyncRepReleaseWaiters(void)
* We're a potential sync standby. Release waiters if there are enough
* sync standbys and we are considered as sync.
*/
+ SyncRepInitWakeList();
+
LWLockAcquire(SyncRepLock, LW_EXCLUSIVE);
/*
@@ -575,6 +586,9 @@ SyncRepReleaseWaiters(void)
LWLockRelease(SyncRepLock);
+ /* wake the released backends now that the lock is down */
+ SyncRepWakeFromList();
+
elog(DEBUG3, "released %d procs up to write %X/%08X, %d procs up to flush %X/%08X, %d procs up to apply %X/%08X",
numwrite, LSN_FORMAT_ARGS(writePtr),
numflush, LSN_FORMAT_ARGS(flushPtr),
@@ -902,13 +916,53 @@ SyncRepGetStandbyPriority(void)
return (SyncRepConfig->syncrep_method == SYNC_REP_PRIORITY) ? priority : 1;
}
+/*
+ * Initialize the list a release collects the procs to latch into, allocating it
+ * the first time this process releases anybody. It is sized for every
+ * backend to be waiting at once; a proc waits in at most one queue, so one
+ * list of that size is enough for a pass over all three.
+ */
+static void
+SyncRepInitWakeList(void)
+{
+ if (SyncRepWakeList.arr == NULL)
+ SyncRepWakeList.arr = (Latch **)
+ MemoryContextAlloc(TopMemoryContext,
+ MaxBackends * sizeof(Latch *));
+}
+
+/*
+ * Set all latches queued to be set.
+ */
+static void
+SyncRepWakeFromList(void)
+{
+ /*
+ * Wake the released backends now that the lock is down. Each latch is a
+ * kill() for a sleeping proc, and running one per released commit inside
+ * the exclusive section makes every committer wait for those syscalls.
+ * The procs below are off the queue with their state already complete, so
+ * nothing here needs the lock's protection. Nothing here can error out
+ * either: only this process dying outright could leave a released proc
+ * completed but unlatched, and the in-lock SetLatch had that same window.
+ * A proc that noticed its state on its own and moved on, even into a new
+ * wait, gets a spurious latch set, which every latch sleeper tolerates.
+ */
+ while (SyncRepWakeList.n > 0)
+ SetLatch(SyncRepWakeList.arr[--SyncRepWakeList.n]);
+}
+
/*
* Walk the specified queue from head. Set the state of any backends that
- * need to be woken, remove them from the queue, and then wake them.
- * Pass all = true to wake whole queue; otherwise, just wake up to
+ * need to be woken and remove them from the queue; the proc's latches to
+ * wake are appended to static wakelist to latch once the lock is down.
+ * Pass all = true to release the whole queue; otherwise, just release up to
* the walsender's LSN.
*
- * The caller must hold SyncRepLock in exclusive mode.
+ * The caller must hold SyncRepLock in exclusive mode, and must set the
+ * latches with SyncRepWakeFromList after releasing it. Unlink, barrier and
+ * state stay together in here: a waiter reads syncRepState without the lock
+ * and must never find itself completed while still on the queue.
*/
static int
SyncRepWakeQueue(bool all, int mode)
@@ -948,10 +1002,9 @@ SyncRepWakeQueue(bool all, int mode)
*/
proc->syncRepState = SYNC_REP_WAIT_COMPLETE;
- /*
- * Wake only when we have set state and removed from queue.
- */
- SetLatch(&(proc->procLatch));
+ /* the list is sized to every process that can ever queue here */
+ Assert(SyncRepWakeList.n < MaxBackends);
+ SyncRepWakeList.arr[SyncRepWakeList.n++] = &proc->procLatch;
numprocs++;
}
@@ -974,6 +1027,8 @@ SyncRepUpdateSyncStandbysDefined(void)
if (sync_standbys_defined !=
((WalSndCtl->sync_standbys_status & SYNC_STANDBY_DEFINED) != 0))
{
+ SyncRepInitWakeList();
+
LWLockAcquire(SyncRepLock, LW_EXCLUSIVE);
/*
@@ -1000,6 +1055,9 @@ SyncRepUpdateSyncStandbysDefined(void)
(sync_standbys_defined ? SYNC_STANDBY_DEFINED : 0);
LWLockRelease(SyncRepLock);
+
+ /* wake the released backends now that the lock is down */
+ SyncRepWakeFromList();
}
else if ((WalSndCtl->sync_standbys_status & SYNC_STANDBY_INIT) == 0)
{
--
2.43.0
From aeaa2b2f2ef29c153399e2f55006e457a87ee08c Mon Sep 17 00:00:00 2001
From: Vadim Ponomarev <[email protected]>
Date: Sat, 15 Aug 2026 12:08:53 +0300
Subject: [PATCH v2 2/5] Compute the synced positions before taking the
sync-rep queue lock
SyncRepReleaseWaiters() takes SyncRepLock and only then walks the
walsender slots to work out the synced write, flush and apply positions.
That walk takes a spinlock per slot, allocates, and for a quorum set sorts
the result, and every cycle of it is spent in the section every committer
lines up on. The comment there conceded the work does not need the lock
and kept it inside anyway, to guarantee the positions are newer than any
previous execution of the routine used.
That guarantee is not needed. The three sites that consume the positions
each move lsn[] forward only when the new reading is ahead of the stored
one, so positions gone stale while the lock was being taken release nobody
and change nothing; a concurrent walsender that got further has already
stored its own.
Compute them before taking the lock, and leave without taking it at all
when this walsender turns out not to be a sync standby.
---
src/backend/replication/syncrep.c | 28 +++++++++++++---------------
1 file changed, 13 insertions(+), 15 deletions(-)
diff --git a/src/backend/replication/syncrep.c b/src/backend/replication/syncrep.c
index 2824cc0e36f..d67d7bd16ad 100644
--- a/src/backend/replication/syncrep.c
+++ b/src/backend/replication/syncrep.c
@@ -518,20 +518,15 @@ SyncRepReleaseWaiters(void)
}
/*
- * We're a potential sync standby. Release waiters if there are enough
- * sync standbys and we are considered as sync.
- */
- SyncRepInitWakeList();
-
- LWLockAcquire(SyncRepLock, LW_EXCLUSIVE);
-
- /*
- * Check whether we are a sync standby or not, and calculate the synced
- * positions among all sync standbys. (Note: although this step does not
- * of itself require holding SyncRepLock, it seems like a good idea to do
- * it after acquiring the lock. This ensures that the WAL pointers we use
- * to release waiters are newer than any previous execution of this
- * routine used.)
+ * We're a potential sync standby. Check whether we are a sync standby
+ * and calculate the synced positions among all sync standbys before
+ * taking the lock: the walk over the walsender slots takes their
+ * spinlocks, allocates, and possibly sorts, and doing all of it under
+ * SyncRepLock delays every committer.
+ *
+ * Positions gone stale by the time the lock is held cost nothing. The
+ * guards further down only ever move lsn[] forward, so a reading older
+ * than a concurrent walsender's simply releases nobody.
*/
got_recptr = SyncRepGetSyncRecPtr(&writePtr, &flushPtr, &applyPtr, &am_sync);
@@ -559,11 +554,14 @@ SyncRepReleaseWaiters(void)
*/
if (!got_recptr || !am_sync)
{
- LWLockRelease(SyncRepLock);
announce_next_takeover = !am_sync;
return;
}
+ SyncRepInitWakeList();
+
+ LWLockAcquire(SyncRepLock, LW_EXCLUSIVE);
+
/*
* Set the lsn first so that when we wake backends they will release up to
* this location.
--
2.43.0
From bbe58ba3ae71ad8ae34d149689df6ed1413ac206 Mon Sep 17 00:00:00 2001
From: Vadim Ponomarev <[email protected]>
Date: Sat, 15 Aug 2026 12:12:19 +0300
Subject: [PATCH v2 3/5] Release the sync-rep waiters once per drained batch of
standby replies
ProcessStandbyReplyMessage() calls SyncRepReleaseWaiters() for every reply
it processes, and several replies routinely sit in the walsender's socket
together. Each of those calls takes SyncRepLock exclusively, so a batch of
replies costs the committers one period of that lock apiece -- computed,
for all but the last reply, from positions the next message in the same
batch immediately makes stale.
Have a reply only mark a release as pending, and run one release at the end
of the drain. The positions in shared memory are the newest of the batch
by then, so the single pass releases everything the individual passes would
have.
A deferred release must survive every way out of the drain, because the
positions the reply already stored are valid whatever follows and the
committers it acknowledged have no other process to wake them:
- the standby's goodbye, an EOF, an invalid message type and an unexpected
message type each run the pending release before leaving. A clean
standby shutdown sends its final reply and the goodbye back to back,
which makes that exit the routine one rather than the exotic one.
- an error thrown while a later message in the same drain is parsed -- a
torn message above all -- leaves through WalSndErrorCleanup(), which runs
the pending release after the locks are dropped.
The test makes both coincidences certain instead of likely. For the first,
a paused standby holds a remote_apply committer in the queue, the walsender
is held with SIGSTOP while the standby applies past the commit and shuts
down, and the released walsender drains the final reply and the goodbye in
one pass. For the second, an injection point right after a drained reply
stands in for the torn message; it fires on every reply, so the walreceiver
is held with SIGSTOP while replay proceeds from WAL already on standby
disk, which makes the first reply after release the one carrying the apply
position the committer waits for. Both halves fail without their fix.
---
src/backend/replication/walsender.c | 59 +++++-
src/test/recovery/meson.build | 1 +
src/test/recovery/t/057_syncrep_release.pl | 206 +++++++++++++++++++++
3 files changed, 264 insertions(+), 2 deletions(-)
create mode 100644 src/test/recovery/t/057_syncrep_release.pl
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index e9331de3df5..89fed06f851 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -94,6 +94,7 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/guc.h"
+#include "utils/injection_point.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/pg_lsn.h"
@@ -206,6 +207,13 @@ static TimestampTz last_reply_timestamp = 0;
/* Have we sent a heartbeat message asking for reply, since last reply? */
static bool waiting_for_ping_response = false;
+/*
+ * Set when a standby reply has updated this walsender's positions and the
+ * waiters those positions release have not been released yet. Raised per
+ * reply, acted on once per drain of the socket.
+ */
+static bool syncrep_release_pending = false;
+
/* Timestamp when walsender received the shutdown request */
static TimestampTz shutdown_request_timestamp = 0;
@@ -300,6 +308,7 @@ static void CreateReplicationSlot(CreateReplicationSlotCmd *cmd);
static void DropReplicationSlot(DropReplicationSlotCmd *cmd);
static void StartReplication(StartReplicationCmd *cmd);
static void StartLogicalReplication(StartReplicationCmd *cmd);
+static void SyncRepFlushPendingRelease(void);
static void ProcessStandbyMessage(void);
static void ProcessStandbyReplyMessage(void);
static void ProcessStandbyHSFeedbackMessage(void);
@@ -381,6 +390,15 @@ WalSndErrorCleanup(void)
pgstat_report_wait_end();
pgaio_error_cleanup();
+ /*
+ * A release deferred by the reply drain survives an error thrown while a
+ * later message in the same drain was being parsed. The positions the
+ * drained reply put in shared memory are valid whatever came after it,
+ * and the committers it acknowledged have no other process to wake them.
+ * The locks are released above, so the queue lock is free to take.
+ */
+ SyncRepFlushPendingRelease();
+
if (xlogreader != NULL && xlogreader->seg.ws_file >= 0)
wal_segment_close(xlogreader);
@@ -2353,6 +2371,22 @@ exec_replication_command(const char *cmd_string)
return true;
}
+/*
+ * Run the release a drained reply deferred. A reply already processed has
+ * put its positions in shared memory, and the committers it acknowledged
+ * have nothing but this process to wake them, so every exit out of the reply
+ * drain runs through here before leaving. The standby's goodbye is the
+ * common one.
+ */
+static void
+SyncRepFlushPendingRelease(void)
+{
+ if (!syncrep_release_pending)
+ return;
+ syncrep_release_pending = false;
+ SyncRepReleaseWaiters();
+}
+
/*
* Process any incoming messages while streaming. Also checks if the remote
* end has closed the connection.
@@ -2379,6 +2413,7 @@ ProcessRepliesIfAny(void)
if (r < 0)
{
/* unexpected error or EOF */
+ SyncRepFlushPendingRelease();
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("unexpected EOF on standby connection")));
@@ -2402,6 +2437,7 @@ ProcessRepliesIfAny(void)
maxmsglen = PQ_SMALL_MESSAGE_LIMIT;
break;
default:
+ SyncRepFlushPendingRelease();
ereport(FATAL,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("invalid standby message type \"%c\"",
@@ -2414,6 +2450,7 @@ ProcessRepliesIfAny(void)
resetStringInfo(&reply_message);
if (pq_getmessage(&reply_message, maxmsglen))
{
+ SyncRepFlushPendingRelease();
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("unexpected EOF on standby connection")));
@@ -2429,6 +2466,7 @@ ProcessRepliesIfAny(void)
*/
case PqMsg_CopyData:
ProcessStandbyMessage();
+ INJECTION_POINT("walsender-reply-drained", NULL);
received = true;
break;
@@ -2450,9 +2488,12 @@ ProcessRepliesIfAny(void)
/*
* PqMsg_Terminate means that the standby is closing down the
- * socket.
+ * socket. The last reply it sent is drained already, and
+ * what that reply acknowledged must not leave with this
+ * process.
*/
case PqMsg_Terminate:
+ SyncRepFlushPendingRelease();
proc_exit(0);
default:
@@ -2468,6 +2509,13 @@ ProcessRepliesIfAny(void)
last_reply_timestamp = last_processing;
waiting_for_ping_response = false;
}
+
+ /*
+ * One release covers every reply drained above: the positions in shared
+ * memory are already the newest ones, and each release takes SyncRepLock
+ * exclusively.
+ */
+ SyncRepFlushPendingRelease();
}
/*
@@ -2498,6 +2546,7 @@ ProcessStandbyMessage(void)
break;
default:
+ SyncRepFlushPendingRelease();
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("unexpected message type \"%c\"", msgtype)));
@@ -2633,8 +2682,14 @@ ProcessStandbyReplyMessage(void)
SpinLockRelease(&walsnd->mutex);
}
+ /*
+ * The release is left for ProcessRepliesIfAny() to run once per drain of
+ * the socket: several replies routinely sit in the buffer together, and
+ * every release takes SyncRepLock exclusively to compute positions this
+ * message has just made stale anyway.
+ */
if (!am_cascading_walsender)
- SyncRepReleaseWaiters();
+ syncrep_release_pending = true;
/*
* Advance our local xmin horizon when the client confirmed a flush.
diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build
index 72113c5ac6e..1e9f37ddcd2 100644
--- a/src/test/recovery/meson.build
+++ b/src/test/recovery/meson.build
@@ -65,6 +65,7 @@ tests += {
't/054_unlogged_sequence_promotion.pl',
't/055_cascade_reconnect.pl',
't/056_standby_snapshot_export.pl',
+ 't/057_syncrep_release.pl',
],
},
}
diff --git a/src/test/recovery/t/057_syncrep_release.pl b/src/test/recovery/t/057_syncrep_release.pl
new file mode 100644
index 00000000000..bc205356fe0
--- /dev/null
+++ b/src/test/recovery/t/057_syncrep_release.pl
@@ -0,0 +1,206 @@
+# Copyright (c) 2026, PostgreSQL Global Development Group
+
+# A walsender coalesces the releases a drained batch of standby replies
+# asks for into one pass at the end of the drain. The drain has early
+# exits, the standby's goodbye being the common one, and a release owed
+# by a reply processed in the same drain must survive them: the positions are
+# in shared memory already, and a committer acknowledged by that reply has
+# nothing else to wake it. What this file proves is that a commit whose
+# ack arrives in the same drain as the standby's goodbye comes back.
+#
+# The choreography makes that coincidence certain instead of likely. A
+# paused standby holds a remote_apply committer in the queue while the
+# flush acks flow; the walsender is then held with SIGSTOP, the standby
+# is resumed, allowed to apply past the commit, and shut down, so its
+# final reply, the one carrying the apply position the committer waits
+# for, lands in the walsender's socket right next to the goodbye. The
+# walsender, released, drains both in one pass.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep time);
+use Test::More;
+
+my $primary = PostgreSQL::Test::Cluster->new('rel_primary');
+$primary->init(allows_streaming => 1);
+$primary->append_conf(
+ 'postgresql.conf', q(
+autovacuum = off
+checkpoint_timeout = 1h
+));
+$primary->start;
+$primary->safe_psql('postgres', 'CREATE TABLE t (id int)');
+$primary->backup('bkp');
+
+# A node streaming from a backup reports its own name as its
+# application_name, which is what the synchronous set goes by.
+my $standby = PostgreSQL::Test::Cluster->new('rel_standby');
+$standby->init_from_backup($primary, 'bkp', has_streaming => 1);
+$standby->start;
+$primary->wait_for_catchup($standby, 'replay');
+
+$primary->safe_psql('postgres',
+ "ALTER SYSTEM SET synchronous_standby_names = 'rel_standby'");
+$primary->reload;
+$primary->poll_query_until('postgres',
+ "SELECT sync_state = 'sync' FROM pg_stat_replication WHERE application_name = 'rel_standby'"
+) or die "standby never became synchronous";
+
+# Hold replay: the flush acks keep flowing, the apply position does not,
+# so a remote_apply commit queues and stays queued.
+$standby->safe_psql('postgres', 'SELECT pg_wal_replay_pause()');
+
+my $committer = $primary->background_psql('postgres');
+$committer->query_until(
+ qr/inserting/, q(
+\echo inserting
+SET synchronous_commit = remote_apply;
+INSERT INTO t VALUES (1);
+));
+
+$primary->poll_query_until('postgres',
+ "SELECT count(*) > 0 FROM pg_stat_activity WHERE wait_event = 'SyncRep'")
+ or die "committer never reached the sync-rep queue";
+my $commit_lsn =
+ $primary->safe_psql('postgres', 'SELECT pg_current_wal_lsn()');
+
+# Hold the walsender, so everything the standby says from here on is
+# drained in one pass.
+my $walsender = $primary->safe_psql('postgres',
+ "SELECT pid FROM pg_stat_replication WHERE application_name = 'rel_standby'"
+);
+die "no walsender pid" unless $walsender =~ /^\d+$/;
+kill 'STOP', $walsender or die "SIGSTOP walsender: $!";
+
+# Let the standby apply past the commit, then say goodbye.
+$standby->safe_psql('postgres', 'SELECT pg_wal_replay_resume()');
+my $deadline = time() + 30;
+while (time() < $deadline)
+{
+ my $replayed = $standby->safe_psql('postgres',
+ "SELECT pg_last_wal_replay_lsn() >= '$commit_lsn'::pg_lsn");
+ last if $replayed eq 't';
+ usleep(100_000);
+}
+$standby->stop('fast');
+
+# The final reply and the goodbye are now side by side in the held
+# walsender's socket. Release it: the drain must not drop the release
+# the reply asks for on its way out.
+kill 'CONT', $walsender or die "SIGCONT walsender: $!";
+
+$deadline = time() + 10;
+my $released = 0;
+while (time() < $deadline)
+{
+ my $waiting = $primary->safe_psql('postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE wait_event = 'SyncRep'");
+ if ($waiting eq '0')
+ {
+ $released = 1;
+ last;
+ }
+ usleep(200_000);
+}
+ok($released,
+ 'a commit acknowledged in the drain the standby left in is released');
+
+# Free the committer session whatever state it is in.
+$primary->safe_psql('postgres',
+ "SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE wait_event = 'SyncRep'"
+) if !$released;
+$committer->quit;
+
+# The other way out of the drain: an error thrown while parsing a later
+# message in the same pass. A reply processed just before it has already
+# asked for a release, and the walsender's error cleanup must run that
+# release on the way out. The connection is lost either way; the
+# committers the reply acknowledged are not. An injection point right
+# after a drained reply stands in for the torn message.
+#
+# The reply the error lands on has to be the one that carries the apply
+# position the committer waits for, and the injection point fires on
+# every drained reply, so no reply may reach the walsender between arming
+# it and the apply position passing the commit. Holding the
+# walreceiver with SIGSTOP is what guarantees that: replay proceeds from
+# WAL already on standby disk, and the receiver's first words on release
+# are the positions as they stand then.
+if (($ENV{enable_injection_points} // 'no') eq 'yes')
+{
+ $standby->start;
+ $primary->wait_for_catchup($standby, 'replay');
+ $primary->safe_psql('postgres', 'CREATE EXTENSION injection_points');
+
+ $standby->safe_psql('postgres', 'SELECT pg_wal_replay_pause()');
+
+ my $committer2 = $primary->background_psql('postgres');
+ $committer2->query_until(
+ qr/inserting2/, q(
+\echo inserting2
+SET synchronous_commit = remote_apply;
+INSERT INTO t VALUES (3);
+));
+ $primary->poll_query_until('postgres',
+ "SELECT count(*) > 0 FROM pg_stat_activity WHERE wait_event = 'SyncRep'"
+ ) or die "second committer never reached the sync-rep queue";
+ my $lsn2 = $primary->safe_psql('postgres', 'SELECT pg_current_wal_lsn()');
+
+ # The commit's WAL must be on standby disk before the receiver is
+ # held, or replay below has nothing to apply.
+ $standby->poll_query_until('postgres',
+ "SELECT pg_last_wal_receive_lsn() >= '$lsn2'::pg_lsn")
+ or die "standby never flushed the commit's WAL";
+
+ my $walreceiver =
+ $standby->safe_psql('postgres', 'SELECT pid FROM pg_stat_wal_receiver');
+ die "no walreceiver pid" unless $walreceiver =~ /^\d+$/;
+ kill 'STOP', $walreceiver or die "SIGSTOP walreceiver: $!";
+
+ # A reply already in flight when the receiver stopped is drained --
+ # and released, long before the injection point is armed.
+ usleep(300_000);
+ $primary->safe_psql('postgres',
+ "SELECT injection_points_attach('walsender-reply-drained', 'error')");
+
+ $standby->safe_psql('postgres', 'SELECT pg_wal_replay_resume()');
+ $deadline = time() + 30;
+ while (time() < $deadline)
+ {
+ my $replayed = $standby->safe_psql('postgres',
+ "SELECT pg_last_wal_replay_lsn() >= '$lsn2'::pg_lsn");
+ last if $replayed eq 't';
+ usleep(100_000);
+ }
+
+ # The receiver's first reply now carries an apply position past the
+ # commit, and the injection point tears the drain right after it.
+ kill 'CONT', $walreceiver or die "SIGCONT walreceiver: $!";
+
+ $deadline = time() + 15;
+ my $released2 = 0;
+ while (time() < $deadline)
+ {
+ my $waiting = $primary->safe_psql('postgres',
+ "SELECT count(*) FROM pg_stat_activity WHERE wait_event = 'SyncRep'"
+ );
+ if ($waiting eq '0')
+ {
+ $released2 = 1;
+ last;
+ }
+ usleep(200_000);
+ }
+ ok($released2,
+ 'a commit acknowledged right before a torn message is released');
+
+ $primary->safe_psql('postgres',
+ "SELECT injection_points_detach('walsender-reply-drained')");
+ $primary->safe_psql('postgres',
+ "SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE wait_event = 'SyncRep'"
+ ) if !$released2;
+ $committer2->quit;
+}
+
+done_testing();
--
2.43.0
From df6e5249d7a1f325a02ea8ef9a3b690e1fe0bef6 Mon Sep 17 00:00:00 2001
From: Vadim Ponomarev <[email protected]>
Date: Sat, 15 Aug 2026 12:14:40 +0300
Subject: [PATCH v2 4/5] Let a committer whose acknowledgement already arrived
skip the queue lock
SyncRepWaitForLSN() runs on every commit that wrote WAL, and it takes
SyncRepLock exclusively before it can find out whether there is anything to
wait for. On a busy primary a large share of those commits find their LSN
already acknowledged and queue for nothing, so the answer costs them a
period of the lock every other committer is lining up on.
Turn lsn[] into an atomic watermark, and read it before taking the lock.
A watermark that already covers the commit's LSN says a valid quorum
acknowledged it, which is exactly the answer the check under the lock
gives. It is only ever moved forward, so a read gone stale can send a
committer to the slow path that would have exited, but never past a wait
it owes.
On platforms where pg_atomic_read_u64() is not a plain load the read is
itself a compare-and-exchange, or a spinlock acquisition where 64-bit
atomics are emulated. Whether the exit still pays for itself there is
untested.
---
src/backend/replication/syncrep.c | 69 ++++++++++++++-------
src/backend/replication/walsender.c | 3 +
src/include/replication/walsender_private.h | 11 +++-
3 files changed, 59 insertions(+), 24 deletions(-)
diff --git a/src/backend/replication/syncrep.c b/src/backend/replication/syncrep.c
index d67d7bd16ad..1b78375e2c9 100644
--- a/src/backend/replication/syncrep.c
+++ b/src/backend/replication/syncrep.c
@@ -134,6 +134,20 @@ static int cmp_lsn(const void *a, const void *b);
static bool SyncRepQueueIsOrderedByLSN(int mode);
#endif
+static inline
+XLogRecPtr
+WalSndCtl_getlsn(int mode)
+{
+ return pg_atomic_read_u64(&WalSndCtl->lsn[mode]);
+}
+
+static inline
+void
+WalSndCtl_setlsn(int mode, XLogRecPtr lsn)
+{
+ pg_atomic_write_u64(&WalSndCtl->lsn[mode], lsn);
+}
+
/*
* ===========================================================
* Synchronous Replication functions for normal user backends
@@ -199,9 +213,32 @@ SyncRepWaitForLSN(XLogRecPtr lsn, bool commit)
Assert(dlist_node_is_detached(&MyProc->syncRepLinks));
Assert(WalSndCtl != NULL);
+ /*
+ * A watermark that already covers this LSN says a valid quorum
+ * acknowledged it, which is the same answer the check below the lock
+ * would give. The watermark only ever moves forward, so a stale read can
+ * only send us to take the lock for nothing, never past a wait we owe.
+ * How often this exit fires depends on the wait mode: it needs the
+ * acknowledgement to have arrived before the committer got here.
+ */
+ if (lsn <= WalSndCtl_getlsn(mode))
+ return;
+
LWLockAcquire(SyncRepLock, LW_EXCLUSIVE);
Assert(MyProc->syncRepState == SYNC_REP_NOT_WAITING);
+ if (lsn <= WalSndCtl_getlsn(mode))
+ {
+ /*
+ * The LSN is older than what we need to wait for. Even if the sync
+ * standby data has not been initialized yet, we are OK to not wait
+ * because we know that there is no point in doing so based on the
+ * LSN.
+ */
+ LWLockRelease(SyncRepLock);
+ return;
+ }
+
/*
* We don't wait for sync rep if SYNC_STANDBY_DEFINED is not set. See
* SyncRepUpdateSyncStandbysDefined().
@@ -211,29 +248,16 @@ SyncRepWaitForLSN(XLogRecPtr lsn, bool commit)
* to be a low cost check.
*
* If the sync standby data has not been initialized yet
- * (SYNC_STANDBY_INIT is not set), fall back to a check based on the LSN,
- * then do a direct GUC check.
+ * (SYNC_STANDBY_INIT is not set), fall back to direct GUC check.
*/
if (WalSndCtl->sync_standbys_status & SYNC_STANDBY_INIT)
{
- if ((WalSndCtl->sync_standbys_status & SYNC_STANDBY_DEFINED) == 0 ||
- lsn <= WalSndCtl->lsn[mode])
+ if ((WalSndCtl->sync_standbys_status & SYNC_STANDBY_DEFINED) == 0)
{
LWLockRelease(SyncRepLock);
return;
}
}
- else if (lsn <= WalSndCtl->lsn[mode])
- {
- /*
- * The LSN is older than what we need to wait for. The sync standby
- * data has not been initialized yet, but we are OK to not wait
- * because we know that there is no point in doing so based on the
- * LSN.
- */
- LWLockRelease(SyncRepLock);
- return;
- }
else if (!SyncStandbysDefined())
{
/*
@@ -566,19 +590,19 @@ SyncRepReleaseWaiters(void)
* Set the lsn first so that when we wake backends they will release up to
* this location.
*/
- if (WalSndCtl->lsn[SYNC_REP_WAIT_WRITE] < writePtr)
+ if (WalSndCtl_getlsn(SYNC_REP_WAIT_WRITE) < writePtr)
{
- WalSndCtl->lsn[SYNC_REP_WAIT_WRITE] = writePtr;
+ WalSndCtl_setlsn(SYNC_REP_WAIT_WRITE, writePtr);
numwrite = SyncRepWakeQueue(false, SYNC_REP_WAIT_WRITE);
}
- if (WalSndCtl->lsn[SYNC_REP_WAIT_FLUSH] < flushPtr)
+ if (WalSndCtl_getlsn(SYNC_REP_WAIT_FLUSH) < flushPtr)
{
- WalSndCtl->lsn[SYNC_REP_WAIT_FLUSH] = flushPtr;
+ WalSndCtl_setlsn(SYNC_REP_WAIT_FLUSH, flushPtr);
numflush = SyncRepWakeQueue(false, SYNC_REP_WAIT_FLUSH);
}
- if (WalSndCtl->lsn[SYNC_REP_WAIT_APPLY] < applyPtr)
+ if (WalSndCtl_getlsn(SYNC_REP_WAIT_APPLY) < applyPtr)
{
- WalSndCtl->lsn[SYNC_REP_WAIT_APPLY] = applyPtr;
+ WalSndCtl_setlsn(SYNC_REP_WAIT_APPLY, applyPtr);
numapply = SyncRepWakeQueue(false, SYNC_REP_WAIT_APPLY);
}
@@ -967,6 +991,7 @@ SyncRepWakeQueue(bool all, int mode)
{
int numprocs = 0;
dlist_mutable_iter iter;
+ XLogRecPtr lsn = WalSndCtl_getlsn(mode);
Assert(mode >= 0 && mode < NUM_SYNC_REP_WAIT_MODE);
Assert(LWLockHeldByMeInMode(SyncRepLock, LW_EXCLUSIVE));
@@ -979,7 +1004,7 @@ SyncRepWakeQueue(bool all, int mode)
/*
* Assume the queue is ordered by LSN
*/
- if (!all && WalSndCtl->lsn[mode] < proc->waitLSN)
+ if (!all && lsn < proc->waitLSN)
return numprocs;
/*
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 89fed06f851..5afcb7d6153 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -4097,7 +4097,10 @@ static void
WalSndShmemInit(void *arg)
{
for (int i = 0; i < NUM_SYNC_REP_WAIT_MODE; i++)
+ {
dlist_init(&(WalSndCtl->SyncRepQueue[i]));
+ pg_atomic_init_u64(&(WalSndCtl->lsn[i]), 0);
+ }
for (int i = 0; i < max_wal_senders; i++)
{
diff --git a/src/include/replication/walsender_private.h b/src/include/replication/walsender_private.h
index b0c80deeb24..964dfcf2b5f 100644
--- a/src/include/replication/walsender_private.h
+++ b/src/include/replication/walsender_private.h
@@ -16,6 +16,7 @@
#include "lib/ilist.h"
#include "nodes/nodes.h"
#include "nodes/replnodes.h"
+#include "port/atomics.h"
#include "replication/syncrep.h"
#include "storage/condition_variable.h"
#include "storage/shmem.h"
@@ -91,9 +92,15 @@ typedef struct
/*
* Current location of the head of the queue. All waiters should have a
- * waitLSN that follows this value. Protected by SyncRepLock.
+ * waitLSN that follows this value. It is atomic, but protected by
+ * SyncRepLock for concurrent writting since it's change triggers wake of
+ * waiters.
+ *
+ * A committer whose LSN this mirror already covers was acknowledged by a
+ * valid quorum and has nothing to wait for, so it reads this before
+ * taking SyncRepLock at all.
*/
- XLogRecPtr lsn[NUM_SYNC_REP_WAIT_MODE];
+ pg_atomic_uint64 lsn[NUM_SYNC_REP_WAIT_MODE];
/*
* Status of data related to the synchronous standbys. Waiting backends
--
2.43.0
From 0afc0988f82992e1b17f92bf176bbcb50969706e Mon Sep 17 00:00:00 2001
From: Yura Sokolov <[email protected]>
Date: Thu, 17 Sep 2026 20:55:01 +0300
Subject: [PATCH v2 5/5] Add stream regress test for synchronous replication
It will allow to experiment with synchronous replication code more wildly.
027_stream_regress_sync.pl is a copy of 027_stream_regress.pl with couple
of changes:
- synchronous_commit is set to remote_apply
- second standby added and checked
- both standbys are in synchronous_standby_names
- rely on remote_apply for standbys to catchup.
---
src/test/recovery/meson.build | 1 +
.../recovery/t/027_stream_regress_sync.pl | 232 ++++++++++++++++++
src/test/regress/expected/stats.out | 2 +-
src/test/regress/sql/stats.sql | 2 +-
4 files changed, 235 insertions(+), 2 deletions(-)
create mode 100644 src/test/recovery/t/027_stream_regress_sync.pl
diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build
index 1e9f37ddcd2..3d46a84c3d6 100644
--- a/src/test/recovery/meson.build
+++ b/src/test/recovery/meson.build
@@ -36,6 +36,7 @@ tests += {
't/025_stuck_on_old_timeline.pl',
't/026_overwrite_contrecord.pl',
't/027_stream_regress.pl',
+ 't/027_stream_regress_sync.pl',
't/028_pitr_timelines.pl',
't/029_stats_restart.pl',
't/030_stats_cleanup_replica.pl',
diff --git a/src/test/recovery/t/027_stream_regress_sync.pl b/src/test/recovery/t/027_stream_regress_sync.pl
new file mode 100644
index 00000000000..06a21e7a8df
--- /dev/null
+++ b/src/test/recovery/t/027_stream_regress_sync.pl
@@ -0,0 +1,232 @@
+
+# Copyright (c) 2024-2026, PostgreSQL Global Development Group
+
+# Run the standard regression tests with streaming replication
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+use File::Basename;
+
+# Initialize primary node
+my $node_primary = PostgreSQL::Test::Cluster->new('primary');
+$node_primary->init(allows_streaming => 1);
+
+# Increase some settings that Cluster->new makes too low by default.
+$node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
+$node_primary->append_conf('postgresql.conf',
+ 'max_prepared_transactions = 10');
+
+# Enable pg_stat_statements to force tests to do query jumbling.
+# pg_stat_statements.max should be large enough to hold all the entries
+# of the regression database.
+$node_primary->append_conf(
+ 'postgresql.conf',
+ qq{shared_preload_libraries = 'pg_stat_statements'
+pg_stat_statements.max = 50000
+compute_query_id = 'regress'
+});
+
+# We'll stick with Cluster->new's small default shared_buffers, but since that
+# makes synchronized seqscans more probable, it risks changing the results of
+# some test queries. Disable synchronized seqscans to prevent that.
+$node_primary->append_conf('postgresql.conf', 'synchronize_seqscans = off');
+
+# Force remote_apply commit mode to toughtest test
+$node_primary->append_conf('postgresql.conf', 'synchronous_commit = remote_apply');
+
+# WAL consistency checking is resource intensive so require opt-in with the
+# PG_TEST_EXTRA environment variable.
+if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ m/\bwal_consistency_checking\b/)
+{
+ $node_primary->append_conf('postgresql.conf',
+ 'wal_consistency_checking = all');
+}
+
+$node_primary->start;
+is( $node_primary->psql(
+ 'postgres',
+ qq[SELECT pg_create_physical_replication_slot('standby_1');]),
+ 0,
+ 'physical slot 1 created on primary');
+is( $node_primary->psql(
+ 'postgres',
+ qq[SELECT pg_create_physical_replication_slot('standby_2');]),
+ 0,
+ 'physical slot 2 created on primary');
+my $backup_name = 'my_backup';
+
+# Take backup
+$node_primary->backup($backup_name);
+
+# Create streaming standby linking to primary
+my $node_standby_1 = PostgreSQL::Test::Cluster->new('standby_1');
+$node_standby_1->init_from_backup($node_primary, $backup_name,
+ has_streaming => 1);
+$node_standby_1->append_conf('postgresql.conf',
+ "primary_slot_name = standby_1");
+$node_standby_1->append_conf('postgresql.conf',
+ 'max_standby_streaming_delay = 600s');
+$node_standby_1->start;
+
+my $node_standby_2 = PostgreSQL::Test::Cluster->new('standby_2');
+$node_standby_2->init_from_backup($node_primary, $backup_name,
+ has_streaming => 1);
+$node_standby_2->append_conf('postgresql.conf',
+ "primary_slot_name = standby_2");
+$node_standby_2->append_conf('postgresql.conf',
+ 'max_standby_streaming_delay = 600s');
+$node_standby_2->start;
+
+$node_primary->safe_psql('postgres',
+ "ALTER SYSTEM SET synchronous_standby_names = 'standby_1,standby_2';");
+$node_primary->reload;
+
+my $dlpath = dirname($ENV{REGRESS_SHLIB});
+my $outputdir = $PostgreSQL::Test::Utils::tmp_check;
+
+# Run the regression tests against the primary.
+my $extra_opts = $ENV{EXTRA_REGRESS_OPTS} || "";
+command_ok(
+ [
+ $ENV{PG_REGRESS},
+ split(' ', $extra_opts),
+ "--dlpath=$dlpath",
+ '--bindir=',
+ '--host=' . $node_primary->host,
+ '--port=' . $node_primary->port,
+ '--schedule=../regress/parallel_schedule',
+ '--max-concurrent-tests=20',
+ '--inputdir=../regress',
+ "--outputdir=$outputdir"
+ ],
+ 'regression tests pass');
+
+my $primary_alive = $node_primary->is_alive;
+my $standby1_alive = $node_standby_1->is_alive;
+my $standby2_alive = $node_standby_2->is_alive;
+is($primary_alive, 1, 'primary alive after regression test run');
+is($standby1_alive, 1, 'standby_1 alive after regression test run');
+is($standby2_alive, 1, 'standby_2 alive after regression test run');
+
+# Clobber all sequences with their next value, so that we don't have
+# differences between nodes due to caching.
+$node_primary->psql('regression',
+ "select setval(seqrelid, nextval(seqrelid)) from pg_sequence");
+
+# No need to wait for standby to catch up because of remote_apply mode.
+
+# Perform a logical dump of primary and standby, and check that they match
+command_ok(
+ [
+ 'pg_dumpall',
+ '--file' => $outputdir . '/primary.dump',
+ '--no-sync', '--no-statistics',
+ '--restrict-key' => 'test',
+ '--port' => $node_primary->port,
+ '--no-unlogged-table-data', # if unlogged, standby has schema only
+ ],
+ 'dump primary server');
+command_ok(
+ [
+ 'pg_dumpall',
+ '--file' => $outputdir . '/standby1.dump',
+ '--no-sync', '--no-statistics',
+ '--restrict-key' => 'test',
+ '--port' => $node_standby_1->port,
+ ],
+ 'dump standby_1 server');
+command_ok(
+ [
+ 'pg_dumpall',
+ '--file' => $outputdir . '/standby2.dump',
+ '--no-sync', '--no-statistics',
+ '--restrict-key' => 'test',
+ '--port' => $node_standby_2->port,
+ ],
+ 'dump standby_2 server');
+compare_files(
+ $outputdir . '/primary.dump',
+ $outputdir . '/standby1.dump',
+ 'compare primary and standby_1 dumps');
+compare_files(
+ $outputdir . '/primary.dump',
+ $outputdir . '/standby2.dump',
+ 'compare primary and standby_2 dumps');
+
+# Likewise for the catalogs of the regression database, after disabling
+# autovacuum to make fields like relpages stop changing.
+$node_primary->append_conf('postgresql.conf', 'autovacuum = off');
+$node_primary->restart;
+command_ok(
+ [
+ 'pg_dump',
+ '--schema' => 'pg_catalog',
+ '--file' => $outputdir . '/catalogs_primary.dump',
+ '--no-sync',
+ '--restrict-key' => 'test',
+ '--port', $node_primary->port,
+ '--no-unlogged-table-data',
+ 'regression',
+ ],
+ 'dump catalogs of primary server');
+command_ok(
+ [
+ 'pg_dump',
+ '--schema' => 'pg_catalog',
+ '--file' => $outputdir . '/catalogs_standby1.dump',
+ '--no-sync',
+ '--restrict-key' => 'test',
+ '--port' => $node_standby_1->port,
+ 'regression',
+ ],
+ 'dump catalogs of standby_1 server');
+command_ok(
+ [
+ 'pg_dump',
+ '--schema' => 'pg_catalog',
+ '--file' => $outputdir . '/catalogs_standby2.dump',
+ '--no-sync',
+ '--restrict-key' => 'test',
+ '--port' => $node_standby_2->port,
+ 'regression',
+ ],
+ 'dump catalogs of standby_2 server');
+compare_files(
+ $outputdir . '/catalogs_primary.dump',
+ $outputdir . '/catalogs_standby1.dump',
+ 'compare primary and standby_1 catalog dumps');
+compare_files(
+ $outputdir . '/catalogs_primary.dump',
+ $outputdir . '/catalogs_standby2.dump',
+ 'compare primary and standby_2 catalog dumps');
+
+# Check some data from pg_stat_statements.
+$node_primary->safe_psql('postgres', 'CREATE EXTENSION pg_stat_statements');
+# This gathers data based on the first characters for some common query types,
+# checking that reports are generated for SELECT, DMLs, and DDL queries with
+# CREATE.
+my $result = $node_primary->safe_psql(
+ 'postgres',
+ qq{WITH select_stats AS
+ (SELECT upper(substr(query, 1, 6)) AS select_query
+ FROM pg_stat_statements
+ WHERE upper(substr(query, 1, 6)) IN ('SELECT', 'UPDATE',
+ 'INSERT', 'DELETE',
+ 'CREATE'))
+ SELECT select_query, count(select_query) > 1 AS some_rows
+ FROM select_stats
+ GROUP BY select_query ORDER BY select_query;});
+is( $result, qq(CREATE|t
+DELETE|t
+INSERT|t
+SELECT|t
+UPDATE|t), 'check contents of pg_stat_statements on regression database');
+
+$node_standby_1->stop;
+$node_standby_2->stop;
+$node_primary->stop;
+
+done_testing();
diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out
index 8b15471248b..4b2f2a50882 100644
--- a/src/test/regress/expected/stats.out
+++ b/src/test/regress/expected/stats.out
@@ -1669,7 +1669,7 @@ SELECT current_setting('fsync') = 'off'
SELECT sum(writes) AS writes, sum(fsyncs) AS fsyncs
FROM pg_stat_io
WHERE context = 'normal' AND object = 'wal' \gset io_sum_wal_normal_after_
-SELECT current_setting('synchronous_commit') = 'on';
+SELECT current_setting('synchronous_commit') IN ('on', 'remote_apply');
?column?
----------
t
diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql
index 674637e172b..c69b5db4af1 100644
--- a/src/test/regress/sql/stats.sql
+++ b/src/test/regress/sql/stats.sql
@@ -769,7 +769,7 @@ SELECT current_setting('fsync') = 'off'
SELECT sum(writes) AS writes, sum(fsyncs) AS fsyncs
FROM pg_stat_io
WHERE context = 'normal' AND object = 'wal' \gset io_sum_wal_normal_after_
-SELECT current_setting('synchronous_commit') = 'on';
+SELECT current_setting('synchronous_commit') IN ('on', 'remote_apply');
SELECT :io_sum_wal_normal_after_writes > :io_sum_wal_normal_before_writes;
SELECT current_setting('fsync') = 'off'
OR current_setting('wal_sync_method') IN ('open_sync', 'open_datasync')
--
2.43.0
