Hi hackers, Was testing a rebased Windows patch using GitHub CI and had some flaky slow test timeout errors for an unrelated pg_waldump test. On Windows (and only Windows) it was running for 500+ seconds and timing out. After a bit of research, I figured it's because of the notoriously slow spoofed IPC that perl does for the forked pg_waldump invocations (that then gets grepped by perl).
001_basic.pl dumps the full WAL range, segments and records, a multiple times per scenario across multiple scenarios, and splits the ~9 MB of output in Perl each time. The attached decodes the full range once against the plain data directory, so every rmgr's desc output is still processed at least once. The fall-off-the-end checks start at the contrecord LSN, a few records from the end. Everything else gets --limit, which counts after filtering, so the --rmgr, --fullpage, --fork, --relation and --block runs stop at the first few matches. This cuts down the IPC considerably. While it was written to speed up Windows, it's faster on Linux too. There's not much to save though as it was already only a couple seconds. On a quiet Linux box of master vs this patch, meson test pg_waldump/001_basic, 10 runs each: 4.43s (4.18-4.65) -> 2.88s (2.49-3.13) Windows should gain a lot more. I only have a few GitHub CI numbers to compare against, but I'm seeing 300-500s for master down to ~40-90s for the patch. Here's a recent 538s run on master: https://github.com/postgres/postgres/actions/runs/33207297796/job/98971778299#step:14:262 Here's the patched version (72s): https://github.com/sehrope/postgres/actions/runs/33228440369/job/99036914098#step:14:199 Note that the bare --stats run has --limit option in front of it (rather than as the last option) on purpose as the reverse failed on Windows. That's due to a bug in src/port/getopt_long.c handling for optional arguments. I'm putting together a separate patch for that. Regards, -- Sehrope Sarkuni Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
0001-Speed-up-pg_waldump-TAP-test.patch
Description: Binary data
