Hi hackers,

I used Opus to analyze the new features in PG 19.
It found that pg_verifybackup never finishes on a tar-format
backup taken with pg_basebackup -Ft -z.

I had manually proved that by hand.

On REL_19_STABLE:

  pg_basebackup -D /tmp/verify_targz.66809/bk -c fast -Ft -z
  pg_verifybackup /tmp/verify_targz.66809/bk
  # still running after 20 s, killing pg_waldump
  pg_verifybackup: error: WAL parsing failed for timeline 1
  pg_verifybackup -n /tmp/verify_targz.66809/bk
  backup successfully verified
  rewrite pg_wal.tar.gz as a single gzip member
  pg_verifybackup /tmp/verify_targz.66809/bk
  backup successfully verified

With WAL streaming, pg_basebackup ends the gzip stream before and
after every tar header, so pg_wal.tar.gz is a chain of gzip members.
astreamer_gzip_decompressor_content() loops until avail_in is zero.
After the first member inflate() keeps returning Z_STREAM_END and
takes no more input, so the loop never ends. The loop runs in
pg_waldump, and pg_verifybackup waits for it.

Patches attached. 0001 calls inflateReset() when inflate() returns
Z_STREAM_END, so the next member is read. 0002 adds a --gzip case to
the tar test in 007_wal.pl and is optional.

Thanks,
Shihao

Attachment: v1-0001-Handle-concatenated-gzip-members-in-astreamer-gzi.patch
Description: Binary data

Attachment: v1-0002-Add-test-for-pg_verifybackup-on-a-gzip-compressed.patch
Description: Binary data

Reply via email to