Hi Fujii-san, Sai Pranav,
Thank you both for checking that. I agree that relying on COPY
batching and shared_buffers
makes the test too sensitive to internal heuristics, so I have removed the test
change in the attached v4 patch.
One alternative I was wondering about: in 011_ib_truncation.pl, after taking
the first incremental backup (where table t is truncated from 8 blocks
to 7 blocks),
we could append one block of zeroes directly to the relation file on disk from
Perl before taking a second incremental backup:
my $relpath =
$primary->safe_psql('postgres', "SELECT pg_relation_filepath('t');");
my $segfile = $primary->data_dir . "/$relpath";
$segfile .= ".1" if -f "$segfile.1";
open(my $fh, '>>:raw', $segfile) or die "open $segfile: $!";
print $fh ("\0" x $block_size);
close($fh);
That simulates an unwritten zero-filled page at block 7 without relying on COPY
or shared_buffers, and deterministically reproduces the bug (returning 8 rows
instead of 7 without the fix).
Do you think a test like that is worth adding, or is it better to leave out the
test and go with v4 as-is?
I am okay with either option; I'm just sharing my thoughts on adding a test
without assuming any internal huristics.
Thanks.
Regards,
Rajesh Kumar
On Mon, Nov 16, 2026 at 11:33 AM Raja Sai pranav <[email protected]>
wrote:
> Hi Fujii-san,
>
> I checked this by running
> https://www.google.com/url?q=http://011_ib_truncation.pl&source=gmail&ust=1794895380937000&sa=E
> against current master without the reconstruct.c change, varying $copy_rows
> (1, 4, 16, 17, 18, 32, 100) and shared_buffers (8MB, 16MB, 128MB). The new
> subtest catches the bug in only 6 of the 21 combinations (copy_rows 16-17
> with 8MB, 17-18 with 16MB and 128MB); in all the others it passes without
> the fix. So I agree it depends too much on COPY's batching to be a reliable
> regression test, and dropping it makes sense, since the reproducer upthread
> already demonstrates the bug and the fix.
>
> Regards,
> Sai Pranav
>
> On Mon, Nov 16, 2026 10:42 AM, Fujii Masao <[email protected]> wrote:
>
>> On Sun, Nov 8, 2026 at 12:21 AM Raja Sai pranav
>> <[email protected]> wrote:
>> > - v3 now adds the bulk-extension regression test I'd flagged as
>> missing: the
>> > node3 / incr2-incr4 chain that bumps shared_buffers so COPY
>> bulk-extends the
>> > relation past an intermediate truncation point, leaving zero-filled
>> unwritten
>> > pages in the truncated range.
>>
>> I just wonder if the added test might be fragile. It relies on internal
>> logics, such as COPY's batching behavior ($copy_rows = 17), buffer pin
>> limits (shared_buffers = 8MB), and the bulk-extension strategy leaving
>> pages unwritten. Changes to any of these could easily make the test pass
>> even without the fix. No?
>>
>> If so, isn't it better to leave out the test for now? Also because
>> the reproducer has already demonstrated the bug and the fix.
>>
>> Regards,
>>
>>
>> --
>> Fujii Masao
>>
>
From 2a9c12d89c3597c3f30954909464840ab8f227da Mon Sep 17 00:00:00 2001
From: Rajesh Kumar Thandapani <[email protected]>
Date: Mon, 16 Nov 2026 13:12:36 +0000
Subject: [PATCH v4] Fix pg_combinebackup with truncated intermediate backups.
When reconstructing a relation from a chain of backups,
reconstruct_from_incremental_file() walks backward from the latest
incremental backup to the full backup, filling sourcemap[] for any
block b < latest_source->truncation_block_length that has not yet been
assigned a source.
However, if an intermediate incremental backup truncated the relation
(s->truncation_block_length < latest_source->truncation_block_length)
and a later backup extended the relation again without writing every
block in the truncated range (for example, via bulk relation extension),
those unwritten blocks still had sourcemap[b] == NULL when the loop
moved past the intermediate backup. As a result, they were sourced from
an earlier backup taken before the truncation, resurrecting stale rows.
Fix by tracking the current truncation_block_length upper bound as we
walk backward through the backup chain and tightening it whenever an
incremental backup has a smaller s->truncation_block_length.
Author: Rajesh Kumar Thandapani <[email protected]>
Reviewed-by: Raja Sai Pranav <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Bug: #19776
Reported-by: Ke <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/bin/pg_combinebackup/reconstruct.c | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/src/bin/pg_combinebackup/reconstruct.c b/src/bin/pg_combinebackup/reconstruct.c
index 5900c5b..f33ad2a 100644
--- a/src/bin/pg_combinebackup/reconstruct.c
+++ b/src/bin/pg_combinebackup/reconstruct.c
@@ -105,6 +105,7 @@ reconstruct_from_incremental_file(char *input_filename,
rfile **sourcemap;
off_t *offsetmap;
unsigned block_length;
+ unsigned truncation_block_length;
unsigned sidx = n_prior_backups;
bool full_copy_possible = true;
int copy_source_index = -1;
@@ -128,6 +129,7 @@ reconstruct_from_incremental_file(char *input_filename,
latest_source = make_incremental_rfile(input_filename);
source[n_prior_backups] = latest_source;
block_length = find_reconstructed_block_length(latest_source);
+ truncation_block_length = latest_source->truncation_block_length;
/*
* For each block in the output file, we need to know from which file we
@@ -224,7 +226,7 @@ reconstruct_from_incremental_file(char *input_filename,
* later backup is one that we can't validate.
*/
blocklength = sb.st_size / BLCKSZ;
- for (b = 0; b < latest_source->truncation_block_length; ++b)
+ for (b = 0; b < truncation_block_length; ++b)
{
if (sourcemap[b] == NULL && b < blocklength)
{
@@ -264,7 +266,7 @@ reconstruct_from_incremental_file(char *input_filename,
{
BlockNumber b = s->relative_block_numbers[i];
- if (b < latest_source->truncation_block_length &&
+ if (b < truncation_block_length &&
sourcemap[b] == NULL)
{
sourcemap[b] = s;
@@ -278,6 +280,19 @@ reconstruct_from_incremental_file(char *input_filename,
full_copy_possible = false;
}
}
+
+ /*
+ * Blocks at or beyond this incremental file's truncation_block_length
+ * were either not present or truncated away before this backup, so
+ * their contents in any earlier backup are stale and must not be
+ * used. This also prevents copying an earlier full file whole even if
+ * its size happens to match latest_source->truncation_block_length.
+ */
+ if (s->truncation_block_length < truncation_block_length)
+ {
+ truncation_block_length = s->truncation_block_length;
+ full_copy_possible = false;
+ }
}
/*
--
2.56.1.703.gf848ed807c-goog