Hello hackers,

This patch improves full-file base backup read performance. The idea is that 
since base backup reads use sequential offsets we can benefit by OS-level 
prefetching. On Linux, the POSIX_FADV_SEQUENTIAL hint increases the FS-level 
readahead window and can improve throughput for read-bound backups.

We do not apply the logic to incremental backups case, since the sequential 
advice may be detrimental to performance due to the potentially random read 
access pattern.

I have benchmarked this patch on an Amazon EC2 instance of type c5.4xlarge with 
a gp3 EBS volume with performance limits of 512MB/s and 4000 IOPS. The OS is 
Ubuntu 24.04 and the filesystem is ext4. The default readahead of the block 
device is 128KB at the OS level.

I built the dataset with pgbench with a scale factor of 100 and performed some 
additional simple updates for 100 seconds which resulted in a PGDATA directory 
size of 2591MiB:
===========================================
/usr/local/pgsql/bin/pgbench -s 100 -i test
/usr/local/pgsql/bin/pgbench --progress=1 -T 100 -j 64 -c 64 
--skip-some-updates test
===========================================
At that point I stopped the postgres server and backed up the dataset (simple 
file copy) for reproducibility.

Before every pg_basebackup run I empty the linux page cache (sync ; echo 3 > 
/proc/sys/vm/drop_caches), start postgres server and time the backup while 
monitoring block device statistics via iostat. WAL path was disabled.


** Vanilla postgres run behaves as follows:
===========================================
$ time /usr/local/pgsql/bin/pg_basebackup --target=blackhole --wal-method=none 
--progress  --checkpoint=fast

real    0m8.029s
===========================================
The effective throughput from statistics divided by time is 197MiB/s.

From block device statistics:
* maximum request size - 128KB
* maximum throughput - 220MiB/s
* average queue depth - 1.8
* average response time - 1.2 ms


** fadvise sequential enabled behaves as follows:
===========================================
$ time /usr/local/pgsql/bin/pg_basebackup --target=blackhole --wal-method=none 
--progress  --checkpoint=fast

real    0m5.813s
===========================================
The effective throughput from statistics divided by time is 272MiB/s.

From block device statistics:
* maximum request size - 256KB
* maximum throughput - 300MiB/s
* average queue depth - 1.8
* average response time - 1.5 ms

The optimization seems to be working as expected, the maximum I/O request size 
is doubled and the backup is 38% faster. Backup performance is not doubled 
though, since there are other bottlenecks in play, e.g. many small files that 
don't allow for large prefetching and potential CPU serialization in the backup 
code.

POSIX_FADV_SEQUENTIAL exactly doubles the block device readahead /proc/ entry 
in linux, which is 128KB by default. ext4 uses this configuration entry and the 
observed behaviour matches expectations since maximum READ request size is 
256KB during the backup. I/O queue depth remains the same, so nothing else 
changes in the I/O pattern. Important to note that EBS volumes have a maximum 
I/O size of 256KB so any larger requests would split into smaller ones and 
increase the queue depth as a side-effect.


I did an additional experiment by modifying global filesystem readahead from 
128KB to 4MB as such:
===========================================
echo 4096 > /sys/block/nvme0n1/queue/read_ahead_kb
===========================================

** Vanilla postgres backup with this change performed as follows:
===========================================
postgres@ip-172-31-26-163:~$ time /usr/local/pgsql/bin/pg_basebackup 
--target=blackhole --wal-method=none --progress --checkpoint=fast

real    0m2.891s
===========================================
The effective throughput from statistics divided by time is 548MiB/s.

From block device statistics:
* maximum request size - 256KB
* maximum throughput - 837MiB/s
* maximum queue depth - 18
* maximum response time - 5.5 ms

Increasing the readahead globally to 4MB makes the backup 178% faster. We 
notice that the EBS throughput limits were temporarily exceeded thanks to burst 
credits that EBS has. It is clear that EBS latency demands higher I/O 
concurrency to fully saturate the underlying disk performance capacity. 
However, it is unclear how other (non-backup) workloads may suffer from such an 
increase of readahead globally. This just shows the potential maximum 
performance we can achieve during a simple backup.

In my opinion, this improvement is a net gain and not a trade-off. Doubling the 
OS-level readeahead only for those sequential accesses halves the number of I/O 
requests and doubles the I/O size in the kernel. This means less I/O 
interference and more efficient kernel block I/O, potentially increasing 
performance for all concurrent I/O workloads and not just for the postgres 
backup running.

Attached is the patch written by Matt Blewitt.

--
Markos Fountoulakis
PlanetScale
From 590592ae1c32d5661ed93f3fa2e41dd3c80b963b Mon Sep 17 00:00:00 2001
From: Matt Blewitt <[email protected]>
Date: Tue, 22 Sep 2026 18:34:22 +0100
Subject: [PATCH v1] basebackup: use sequential access advice for full-file
 reads

---
 src/backend/backup/basebackup.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/src/backend/backup/basebackup.c b/src/backend/backup/basebackup.c
index f8c3520e5b8..3819aa65a23 100644
--- a/src/backend/backup/basebackup.c
+++ b/src/backend/backup/basebackup.c
@@ -12,6 +12,7 @@
  */
 #include "postgres.h"
 
+#include <fcntl.h>
 #include <sys/stat.h>
 #include <unistd.h>
 #include <time.h>
@@ -127,6 +128,7 @@ static void perform_base_backup(basebackup_options *opt, 
bbsink *sink,
                                                                
IncrementalBackupInfo *ib);
 static void parse_basebackup_options(List *options, basebackup_options *opt);
 static int     compareWalFileNames(const ListCell *a, const ListCell *b);
+static void basebackup_fadvise_sequential(int fd);
 static ssize_t basebackup_read_file(int fd, char *buf, size_t nbytes, off_t 
offset,
                                                                        const 
char *filename, bool partial_read_ok);
 
@@ -574,6 +576,7 @@ perform_base_backup(basebackup_options *opt, bbsink *sink,
                                                (errcode_for_file_access(),
                                                 errmsg("could not open file 
\"%s\": %m", pathbuf)));
                        }
+                       basebackup_fadvise_sequential(fd);
 
                        if (fstat(fd, &statbuf) != 0)
                                ereport(ERROR,
@@ -1625,6 +1628,8 @@ sendFile(bbsink *sink, const char *readfilename, const 
char *tarfilename,
                                (errcode_for_file_access(),
                                 errmsg("could not open file \"%s\": %m", 
readfilename)));
        }
+       if (incremental_blocks == NULL)
+               basebackup_fadvise_sequential(fd);
 
        /*
         * Non-relation files may be replaced by a rename(), making irrelevant 
the
@@ -2204,6 +2209,16 @@ convert_link_to_directory(const char *pathbuf, struct 
stat *statbuf)
                statbuf->st_mode = S_IFDIR | pg_dir_create_mode;
 }
 
+static void
+basebackup_fadvise_sequential(int fd)
+{
+#if defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_SEQUENTIAL)
+       (void) posix_fadvise(fd, 0, 0, POSIX_FADV_SEQUENTIAL);
+#else
+       (void) fd;
+#endif
+}
+
 /*
  * Read some data from a file, setting a wait event and reporting any error
  * encountered.
-- 
2.53.0

Reply via email to