> Couldn't we do something like have another version of DropRelationBuffers() > which accepts a parameter for the old number of blocks and uses that instead > of calling smgrnblocks_cached()?
Hi David, Here is a patch for that idea. It only covers relation truncation, such as the tail truncation done by vacuum. DROP TABLE and plain TRUNCATE go through DropRelationsAllBuffers() and are not changed. smgrtruncate() already gets the current fork sizes from its callers as old_nblocks. The patch passes them on to DropRelationBuffers(), so no lseek() is added. It also adds an assertion to RelationTruncate() that the caller holds AccessExclusiveLock. In the original thread Tom said there is no room for "good enough" here, because a dirty buffer left behind breaks the checkpointer. This is why the passed sizes are safe. 1. Nothing can extend the relation between measuring and dropping. Outside recovery the caller holds AccessExclusiveLock. During recovery only the startup process extends relations, and the passed sizes equal the cached ones, so recovery does not change. 2. If the size we pass is larger than the real one, we only do some useless lookups. If it were smaller, we would miss buffers past it. That can only happen if the relation grows after we measure it, and point 1 rules that out. 3. Buffers can exist past the end of the file after a failed extension. They are neither valid nor dirty, so nobody writes them, and skipping them is harmless. One case does differ from today. Reading past the end of the file with zero_damaged_pages can leave a valid zero page behind. If the data is already corrupt and someone writes to that page, the full scan would drop it. The patch leaves it for the checkpointer. I think that is fine for a setting meant for recovering from corruption. To look for missed buffers, I used a temporary check that scanned the whole pool after each targeted drop. It caught deliberately wrong sizes, and found nothing in the full test suite or in a concurrent stress run. Numbers come from release builds on the same base commit, timing only the DropRelationBuffers() call. Values are medians of 8 truncations. Truncating 2000 pages, with 8 clients running pgbench -S: shared_buffers master patched 2GB 1.83 ms 0.24 ms 8GB 6.29 ms 0.18 ms 32GB 23.35 ms 0.27 ms Truncating 30000 pages with 8GB, which is below the scan threshold of 32768 blocks: background load master patched none 7.24 ms 2.72 ms 32 clients, -S 9.16 ms 4.06 ms 8 clients, TPC-B 8.66 ms 3.49 ms Above the threshold both sides do the same full scan. With 2GB and 30000 pages, where the threshold is 8192 blocks, both took about 4.6 ms. Background throughput was the same on both sides in every run. Thanks, Shihao
v1-0001-Let-smgrtruncate-pass-the-known-fork-sizes-to-Dro.patch
Description: Binary data
