Hi, On Tue, Aug 25, 2026 at 12:02 AM Bharath Rupireddy <[email protected]> wrote: > > Although fetching from archive could be much faster (with parallel > restoring of a batch of WAL files without going through the > replication protocol), as rightly mentioned upthread, a transiently > failed archive network or storage could lead to the primary's disk > filling up or blocking logical replication, and perhaps vacuum too. > > Can we start by having some tests where the archive is in S3 or a > similar remote store and measure the latencies for fetching say 1000 > or more WAL files from the archive vs the primary via streaming with a > simple restore command, and also the impact on the primary while the > standby stays in archive mode? This helps set some context for the use > case. >
Thanks for the suggestion. I'd like to hold off on measuring archive recovery vs. streaming speed. In my experience (and this seems especially true in cloud env), which one performs better depends heavily on the setup: for some users archive recovery via an optimized restore_command may well outperform streaming, and for others it's the reverse. A benchmark run in one environment is unlikely to generalize, so I'm not sure it would settle anything either way. The main motivation for reviving this thread for me is to set the performance question of archive recovery vs. streaming replication aside entirely and focus instead on the impact of a standby remaining in archive recovery mode for a sustained period specifically, how that affects the primary (via a stalled replication slot and unbounded WAL retention) and, where slot sync is in use, the logical replicas downstream as well. To put this in concrete terms: consider a primary under a high write workload generating WAL at 100 MB/s, or roughly 360 GB per hour. When a new or existing standby starts recovery with a substantial archive backlog, it continues restoring archived WAL while suitable WAL remains available. Because WAL redo is largely sequential and single-process, it may/does not keep pace with WAL generation under such a workload, even on well provisioned hardware. The standby therefore remains in archive recovery for an extended period. As a result, the primary could retain more than 500 GB of WAL within a couple of hours and several terabytes within four or five hours. This can exhaust the primary’s storage and cause an outage, forcing users either to increase storage substantially or accept prolonged unavailability. The effect can also extend to logical replicas when slot synchronization is enabled. When synchronized_standby_slots includes the stalled physical slot, logical WAL senders cannot advance beyond that physical standby’s confirmed position. The resulting delay therefore affects not only the primary and standby, but also logical replicas downstream. We have observed this behavior on several occasions. Although WAL prefetching implemented at our end substantially improves archive retrieval, redo remains largely single-process and still falls behind WAL generation on heavily loaded systems generating 100 MB/s of WAL or more, keeping the standby in archive recovery for an extended period, allowing the associated replication slot and retained WAL on the primary to continue growing. -- With Regards, Ashutosh Sharma.
