Hi all, As was previously discussed at the thread surrounding [1]: Currently any block registration in WAL takes up at least 8 bytes in xlog overhead, regardless of how much data is included in that block: 1 byte for block ID, 1 byte for fork and flags, 2 bytes for the block data length, and 4 bytes for the blockNo. (Usually, another 12 bytes are used for a RelFileLocator; but that may not be included for some blocks in the record when conditions apply)
Attached is a patch that reduces this overhead by up to 2 bytes by encoding how large the block data length field is into the block ID, and thus optionally reducing the block data's length field to 0 bytes. Examples: cross-page update records will now be 2 bytes shorter, because the record never registers any data for the new block of the update; pgbench transactions are now either 6 or 8 bytes smaller depending on whether the update crosses a page boundary (in xlog record size; after alignment it is 0 or 4/8 bytes, depending on MAXALIGN and whether the updates are cross-page updates). It changes the block IDs used to fit in 6 bits, using the upper 2 bits of the block_id field to store how much data is contained in the record (0, <=UINT8_MAX, or <=UINT16_MAX bytes). This is part 1 of a series of patches trying to decrease the size of WAL; see also [0], [1] and [2] for more info on what's still to go. I'm working on a separate, much more involved patch for the XLogRecord header itself, similar to the patch in [1], which I expect to send sometime soon as well. Unless someone thinks the patches should be discussed as one series, I'm planning on posting that in another thread, as I don't see any meaningful dependencies between the patches, and the XLR header patch will be quite a bit larger than this one. Kind regards, Matthias van de Meent Neon, Inc. [0] https://wiki.postgresql.org/wiki/Updating_the_WAL_infrastructure [1] https://www.postgresql.org/message-id/flat/CAEze2Wjd3jY_UhhOGdGGnC6NO%3D%2BNmtNOmd%3DJaYv-v-nwBAiXXA%40mail.gmail.com#17a51d83923f4390d8f407d0d6c5da07 [2] https://www.postgresql.org/message-id/flat/CAEze2Whf%3DfwAj7rosf6aDM9t%2B7MU1w-bJn28HFWYGkz%2Bics-hg%40mail.gmail.com PS. Benchmark results on my system (5950x with other light tasks running) don't show an obviously negative effect in a 10-minute run with these arbitrary pgbench settings on a fresh cluster with default configuration: ./pg_install/bin/pgbench postgres -j 2 -c 6 -T 600 -M prepared [...] master: tps = 375 patched: tps = 381
v1-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch
Description: Binary data
