Hi, Bug #19754 [1] reported an issue with zlib 1.3.2 where gzread() and gzclose() may return incorrect results. The current implementation of pg_restore relies on these two functions, so pg_restore may reject valid dump files or fail to detect truncated gzip streams in some cases. Please read [1] for details if you are interested.
We already have astreamer_gzip, which directly calls zlib's low-level inflate() function to decompress data. So I initially thought about directly reusing astreamer_gzip in compress_gzip.c. However, after some experiments, I realized that its push interface does not fit pg_restore's read interface well. It also reports errors through pg_fatal(), whereas a shared helper should let callers handle errors themselves. So, I extracted the core decompression logic into a separate gzip.c and used it in both astreamer_gzip and compress_gzip.c. To reproduce the issue from the bug report, I created a new TAP test, 007_pg_dump_gzip.pl. When built and run with zlib 1.3.2, the test should fail without this patch: ``` # +++ tap check in src/bin/pg_dump +++ t/007_pg_dump_gzip.pl .. 1/? # Failed test 'restore the intact gzip file whose sizes are on both buffer boundaries' # at t/007_pg_dump_gzip.pl line 102. # ---------- command failed ---------- # pg_restore --file /Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_restore.sql /Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_dump # -------------- stderr -------------- # pg_restore: error: could not close data file "/Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_dump/3875.dat": Undefined error: 0 # ------------------------------------ # Failed test 'fail to restore the truncated gzip data file' # at t/007_pg_dump_gzip.pl line 120. ``` See the attached patch for details: * 0001 - Only a TAP test that reproduce bug #19754 * 0002 - The fix plus more test coverage @Stefan, would you please help verify the fix on your end? [1] https://www.postgresql.org/message-id/[email protected] Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/
v1-0001-Add-a-TAP-test-for-pg_restore-bug-19754.patch
Description: Binary data
v1-0002-Fix-gzip-stream-validation-in-pg_restore.patch
Description: Binary data
