Hi, I had another pass at this and have attached v3, now split into two patches. The removal and recycling cases have one important difference: only recycling exposes a new WAL filename to the write path.
0001 is the straightforward bit. For segments that a checkpoint removes, it
does the unlinks first, fsyncs pg_wal once, and then cleans up the archive
status files. I left the timeline-switch path alone. This is not only for
wal_recycle=off; with recycling enabled, surplus segments still take the
removal path.
I reran the removal benchmark with the default 16 MiB WAL segment size. The
numbers below are for 0001 alone; 0002 was not applied. This was on ext4 with
fsync enabled and wal_recycle=off, so every eligible segment went through the
removal path. Each number is the median of five alternating runs after three
warmups:
files removed master 0001
1 22 ms 23 ms
8 53 ms 32 ms
32 135 ms 64 ms
64 201 ms 97 ms
128 394 ms 154 ms
The one-file case was within run-to-run variation, as expected. Once there
are several files to remove, the per-file directory fsyncs start to show up;
at 128 files the time went from 394 ms to 154 ms, about 60% lower (2.52x).
This is not a TPS result, just checkpoint cleanup time. Since 0001 avoids
the write-path changes in 0002, I thought it was worth splitting out for
separate consideration.
0002 is the awkward bit, and I have left it marked WIP. Going back over v2,
I found a hole in the durable-segment high-water mark. A foreground process
can find the names in the pending recycle batch already occupied, install its
own segment at a higher number, and move the frontier past lower renames that
are not durable yet.
The current patch plugs that hole with an active-batch marker. While a batch
is in flight, XLogFileInit() takes the file-and-directory fsync fallback even
if the frontier says the segment is covered. The new TAP test forces exactly
this interleaving; without the marker check, it times out waiting for the
fallback.
That handles the case I found, but I am still on the fence about 0002. It
feels like a fair amount of plumbing between checkpoint cleanup and the WAL
write path just to save these fsyncs. Is there a cleaner invariant that I am
missing, or a less invasive way to make the batched renames safe?
Regards,
Ayush
v3-0001-Batch-fsyncs-when-removing-old-WAL-segments.patch
Description: Binary data
v3-0002-WIP-Batch-fsyncs-when-recycling-WAL-segments.patch
Description: Binary data
