On 18/05/2023 17:59, Matthias van de Meent wrote:
Attached is a patch that reduces this overhead by up to 2 bytes by
encoding how large the block data length field is into the block ID,
and thus optionally reducing the block data's length field to 0 bytes.
Examples: cross-page update records will now be 2 bytes shorter,
because the record never registers any data for the new block of the
update; pgbench transactions are now either 6 or 8 bytes smaller
depending on whether the update crosses a page boundary (in xlog
record size; after alignment it is 0 or 4/8 bytes, depending on
MAXALIGN and whether the updates are cross-page updates).
It changes the block IDs used to fit in 6 bits, using the upper 2 bits
of the block_id field to store how much data is contained in the
record (0, <=UINT8_MAX, or <=UINT16_MAX bytes).
Perhaps we should introduce a few generic inline functions to do varint
encoding. That could be useful in many places, while this scheme is very
tailored for XLogRecordBlockHeader.
We could replace XLogRecordDataHeaderShort and XLogRecordDataHeaderLong
with this too. With just one XLogRecordDataHeader, with a
variable-length length field.
This is part 1 of a series of patches trying to decrease the size of
WAL; see also [0], [1] and [2] for more info on what's still to go.
I'm working on a separate, much more involved patch for the XLogRecord
header itself, similar to the patch in [1], which I expect to send
sometime soon as well.
Unless someone thinks the patches should be discussed as one series,
I'm planning on posting that in another thread, as I don't see any
meaningful dependencies between the patches, and the XLR header patch
will be quite a bit larger than this one.
Kind regards,
Matthias van de Meent
Neon, Inc.
[0] https://wiki.postgresql.org/wiki/Updating_the_WAL_infrastructure
Good ideas here. Eliminating the two padding bytes from XLogRecord in
particular seems like a pure win.
PS. Benchmark results on my system (5950x with other light tasks
running) don't show an obviously negative effect in a 10-minute run
with these arbitrary pgbench settings on a fresh cluster with default
configuration:
./pg_install/bin/pgbench postgres -j 2 -c 6 -T 600 -M prepared
[...]
master: tps = 375
patched: tps = 381
That was probably not CPU limited, so that any overhead in generating
the WAL would not show up. Try PGOPTIONS="-csynchronous_commit=off" and
pgbench -N option. And make sure the scale is large enough that there is
no lock contention. Also would be good to measure the overhead in
replaying the WAL.
How much space saving does this yield?
- Heikki
