Hi, On Tue, 22 Sept 2026 at 00:38, Zsolt Parragi <[email protected]> wrote: > > Hello! > > Shouldn't 021 get the same fix (0001)? That's only executed with > checksum_extended so its not that visible on CI, but it has the same > issue.
Yes, you are right. I realized that we can do the same thing for all test_checksum tests, so I applied similar change to all of the related checksum tests in 0001. -- Regards, Nazir Bilal Yavuz Microsoft
From 2ad57fce46de1b053039c91248fd127888de09ad Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Tue, 15 Sep 2026 16:28:28 +0300 Subject: [PATCH v3 1/2] Avoid idle WAL waits in checksum tests Use wait_for_replay_catchup() in checksum tests that explicitly wait for the upstream insert LSN. That position can include unrelated WAL not yet flushed and available for streaming on an idle primary. Completed transactions, backups, checkpoints, and the explicit flushes of checksum state changes already cover the WAL these waits need to replay. In the 013_rewind test, also switch WAL on the source before starting the target. This flushes through the minimum recovery point chosen by pg_rewind, avoiding a startup delay while an unflushed full_page_writes change waits for the background writer to log its next snapshot. This reduces test time of 013_rewind test from ~45s to ~3s. Discussion: https://postgr.es/m/CAN55FZ1Yak_xBqMaDQsD7atpBkGLEkF-DKXcs3nLHM1Uq4YRew%40mail.gmail.com --- .../test_checksums/t/003_standby_restarts.pl | 12 ++++-------- .../test_checksums/t/011_standby_straddle.pl | 15 +++++---------- src/test/modules/test_checksums/t/013_rewind.pl | 12 +++++++++--- .../t/021_rewind_divergent_transitions.pl | 8 ++++---- 4 files changed, 22 insertions(+), 25 deletions(-) diff --git a/src/test/modules/test_checksums/t/003_standby_restarts.pl b/src/test/modules/test_checksums/t/003_standby_restarts.pl index b05ffd8643a..d70b90598da 100644 --- a/src/test/modules/test_checksums/t/003_standby_restarts.pl +++ b/src/test/modules/test_checksums/t/003_standby_restarts.pl @@ -42,8 +42,7 @@ $node_primary->safe_psql('postgres', "CREATE TABLE t AS SELECT generate_series(1,10000) AS a;"); # Wait for standby to catch up -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); # Check that checksums are turned off on all nodes test_checksum_state($node_primary, 'off'); @@ -140,8 +139,7 @@ $node_primary->safe_psql( SELECT g, repeat('x', 100) FROM generate_series(1, 1000) g; CREATE INDEX unlogged_promo_payload_idx ON unlogged_promo (payload); ]); -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); # Get the relfilenode and database OID so we can inspect the filesystem my $unlogged_rfn = $node_primary->safe_psql('postgres', @@ -161,8 +159,7 @@ wait_for_checksum_state($node_standby, 'on'); # After standby replays, the unlogged main file must still not exist. # If the bug were present, FPI replay would materialize the full table. -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); ok( !-f "$standby_datadir/base/$db_oid/$unlogged_rfn", 'standby has no main fork for unlogged table after enable'); @@ -181,8 +178,7 @@ is($result, '1000', # Alter persistence to logged, and make sure we can read it on both the primary # and standby without any page verification errors in the logfiles. $node_primary->safe_psql('postgres', 'ALTER TABLE unlogged_tbl SET logged;'); -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); $result = $node_primary->safe_psql('postgres', 'SELECT sum(a) FROM unlogged_tbl;'); diff --git a/src/test/modules/test_checksums/t/011_standby_straddle.pl b/src/test/modules/test_checksums/t/011_standby_straddle.pl index e50fb65fcaf..4d3aa6f1249 100644 --- a/src/test/modules/test_checksums/t/011_standby_straddle.pl +++ b/src/test/modules/test_checksums/t/011_standby_straddle.pl @@ -92,8 +92,7 @@ bgwriter_lru_maxpages = 0 ]); $node_standby->start; -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); test_checksum_state($node_primary, 'off'); test_checksum_state($node_standby, 'off'); @@ -102,8 +101,7 @@ test_checksum_state($node_standby, 'off'); # record reaches the standby until the enabling is released, so this remains # the standby's backup starting checkpoint throughout. $node_primary->safe_psql('postgres', 'CHECKPOINT;'); -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); $node_standby->safe_psql('postgres', 'CHECKPOINT;'); # Put everything the enabling writes into fresh WAL segments, so that the @@ -143,8 +141,7 @@ $node_primary->safe_psql('postgres', # The standby has now replayed the state change: its pg_control says "on" # while the rewritten pages are only dirty in its shared buffers. -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); wait_for_checksum_state($node_standby, 'on'); # Crash the standby, losing the dirty rewritten pages. @@ -197,8 +194,7 @@ isnt($ret, 0, 'standby refuses connections while below the state change'); $node_standby->enable_streaming($node_primary); $node_standby->reload; $node_standby->poll_query_until('postgres', 'SELECT true;'); -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); # The rewritten pages are again only dirty in shared buffers, so the on-disk # pages still lack checksums. A base backup must skip verification entirely @@ -228,8 +224,7 @@ $node_primary->poll_query_until('postgres', # A restartpoint on the final checkpoint lets verification resume, and a # backup started from it must again pass. -$node_primary->wait_for_catchup($node_standby, 'replay', - $node_primary->lsn('insert')); +$node_primary->wait_for_replay_catchup($node_standby); $node_standby->safe_psql('postgres', 'CHECKPOINT;'); $node_standby->command_checks_all( diff --git a/src/test/modules/test_checksums/t/013_rewind.pl b/src/test/modules/test_checksums/t/013_rewind.pl index a791e24317d..21715ace9f3 100644 --- a/src/test/modules/test_checksums/t/013_rewind.pl +++ b/src/test/modules/test_checksums/t/013_rewind.pl @@ -71,7 +71,8 @@ my $node_b = PostgreSQL::Test::Cluster->new('node_b'); $node_b->init_from_backup($node_a, 'backup', has_streaming => 1); $node_b->start; -$node_a->wait_for_catchup($node_b, 'replay', $node_a->lsn('insert')); +# Backup completion has flushed the required WAL. +$node_a->wait_for_replay_catchup($node_b); test_checksum_state($node_a, 'off'); test_checksum_state($node_b, 'off'); @@ -83,7 +84,7 @@ $node_b->safe_psql('postgres', # in a background session; it will block on the injection point with # the checkpointer busy until released. $node_a->safe_psql('postgres', "CHECKPOINT;"); -$node_a->wait_for_catchup($node_b, 'replay', $node_a->lsn('insert')); +$node_a->wait_for_replay_catchup($node_b); my $bg_psql = $node_b->background_psql('postgres', on_error_stop => 0); $bg_psql->query_until( @@ -182,9 +183,14 @@ port = @{[$node_a->port]} primary_conninfo = '$connstr application_name=@{[$node_a->name]}' ]); $node_a->set_standby_mode; + +# Flush WAL through the minimum recovery point chosen by pg_rewind. The +# full_page_writes change can leave an unflushed record on the idle source, +# delaying startup until the background writer logs its next snapshot. +$node_b->safe_psql('postgres', 'SELECT pg_switch_wal();'); $node_a->start; -$node_b->wait_for_catchup($node_a, 'replay', $node_b->lsn('insert')); +$node_b->wait_for_replay_catchup($node_a); test_checksum_state($node_a, 'on'); is($node_a->safe_psql('postgres', "SELECT count(*) FROM t;"), diff --git a/src/test/modules/test_checksums/t/021_rewind_divergent_transitions.pl b/src/test/modules/test_checksums/t/021_rewind_divergent_transitions.pl index 6f95384f76a..e52fcc56ee5 100644 --- a/src/test/modules/test_checksums/t/021_rewind_divergent_transitions.pl +++ b/src/test/modules/test_checksums/t/021_rewind_divergent_transitions.pl @@ -87,7 +87,7 @@ $node_a->backup('backup'); my $node_b = PostgreSQL::Test::Cluster->new('node_b'); $node_b->init_from_backup($node_a, 'backup', has_streaming => 1); $node_b->start; -$node_a->wait_for_catchup($node_b, 'replay', $node_a->lsn('insert')); +$node_a->wait_for_replay_catchup($node_b); # Clean switchover to B; enable checksums online on it. $node_a->stop('fast'); @@ -137,7 +137,7 @@ primary_conninfo = '$connstr_b application_name=@{[$node_a->name]}' $node_a->set_standby_mode; $node_a->start; -$node_b->wait_for_catchup($node_a, 'replay', $node_b->lsn('insert')); +$node_b->wait_for_replay_catchup($node_a); test_checksum_state($node_a, 'on'); is($node_a->safe_psql('postgres', "SELECT count(*) FROM t_div;"), @@ -147,7 +147,7 @@ is($node_a->safe_psql('postgres', "SELECT count(*) FROM t_div;"), # checksums online so the next divergence point carries "off", and let # A replay the change. disable_data_checksums($node_b, wait => 'off'); -$node_b->wait_for_catchup($node_a, 'replay', $node_b->lsn('insert')); +$node_b->wait_for_replay_catchup($node_a); test_checksum_state($node_a, 'off'); # Clean switchover back to A; enable checksums online on it. @@ -184,7 +184,7 @@ primary_conninfo = '$connstr_a application_name=@{[$node_b->name]}' $node_b->set_standby_mode; $node_b->start; -$node_a->wait_for_catchup($node_b, 'replay', $node_a->lsn('insert')); +$node_a->wait_for_replay_catchup($node_b); test_checksum_state($node_b, 'on'); is($node_b->safe_psql('postgres', "SELECT count(*) FROM t;"), -- 2.47.3
From 7dbc4f9445f96a5ace5619f3c6949d2c0c7185d8 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Thu, 17 Sep 2026 13:23:12 +0300 Subject: [PATCH v3 2/2] Fix WAL file selection in 013_rewind test Without an explicit WAL filename, pg_waldump determines the segment size from the first WAL-named directory entry. A preallocated segment with an uninitialized header can make this fail before the checkpoint is read. Pass the WAL filename recorded in backup_label to avoid depending on directory enumeration order. Use command_like() so command failures and stderr are checked rather than appearing only as an empty-output mismatch. Discussion: https://postgr.es/m/CAN55FZ1Yak_xBqMaDQsD7atpBkGLEkF-DKXcs3nLHM1Uq4YRew%40mail.gmail.com --- src/test/modules/test_checksums/t/013_rewind.pl | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/src/test/modules/test_checksums/t/013_rewind.pl b/src/test/modules/test_checksums/t/013_rewind.pl index 21715ace9f3..7e0ba87c8e2 100644 --- a/src/test/modules/test_checksums/t/013_rewind.pl +++ b/src/test/modules/test_checksums/t/013_rewind.pl @@ -151,15 +151,21 @@ $backup_label =~ /^CHECKPOINT LOCATION: ([0-9A-F\/]+)$/m or die "checkpoint location missing from backup_label"; is($1, $shutdown_ckpt, 'replay starts at the switchover checkpoint'); -($stdout, $stderr) = run_command( +# Specify the WAL file so that pg_waldump does not try to determine the +# segment size from an arbitrary, possibly preallocated, file in pg_wal. +$backup_label =~ /^START WAL LOCATION: [0-9A-F\/]+ \(file ([0-9A-F]{24})\)$/m + or die "WAL file name missing from backup_label"; +my $shutdown_wal = $1; + +command_like( [ 'pg_waldump', '-p' => $node_a->data_dir . '/pg_wal', - '-t' => 1, '-s' => $shutdown_ckpt, '-n' => 1, - ]); -like($stdout, qr/CHECKPOINT_SHUTDOWN/, + $shutdown_wal, + ], + qr/CHECKPOINT_SHUTDOWN/, 'last common checkpoint is a shutdown checkpoint'); # pg_rewind keeps the target's own checksum state in the control file it -- 2.47.3
