> On Aug 14, 2026, at 06:59, Chao Li <[email protected]> wrote: > > > >> On Aug 14, 2026, at 04:57, Daniel Gustafsson <[email protected]> wrote: >> >>> On 13 Aug 2026, at 07:11, Chao Li <[email protected]> wrote: >> >>> PFA v7: addressed Jipan’s comment. >> >> Sorry for being slow on this, things are quite busy but I hope to have a >> review >> soon. While poking at this I realized that our compression code in pg_dump >> likely has the same issue. I hacked up a quick PoC diff (attached) but it's >> untested (can one actually test the data-in-zstd-internal-buffers case at >> all?) >> and mainly a sketch. If you want to pick it up and rework into this patchset >> to tackle it treewide then that would be fantastic. >> >> -- >> Daniel Gustafsson >> >> <pg_dump.diff.txt> > > I can work on this today.
I just checked pg_dump/pg_restore. The problem exists only with zstd and lz4, gzip doesn't have the problem. Daniel’s PoC covers the custom-archive-format path, but not the directory-format path. For the directory-format path, we can reproduce the problem by simply truncating one byte from a compressed data file. For the custom-archive path, reproducing the problem is less straightforward because the compressed data is stored inside length-prefixed archive blocks. Simply truncating the file can make archive parsing fail before the decompressor sees the truncated frame. I created a repro script, see the attached shell script. Before the fix, the output contains: ``` custom zstd: exit status 0 directory zstd: exit status 0 custom lz4: exit status 0 directory lz4: exit status 0 pg_restore: error: could not uncompress data: (null) custom gzip: exit status 1 pg_restore: error: could not close data file "/tmp/pgdump-trunc.1rxbf0/gzip-dir-bad/3931.dat": Undefined error: 0 directory gzip: exit status 1 ``` This shows that gzip reports failure, while zstd and lz4 silently accept the truncated dump files. After the fix, zstd and lz4 report failures as well: ``` pg_restore: error: could not decompress data: compressed stream is incomplete custom zstd: exit status 1 pg_restore: error: could not decompress data: compressed stream is incomplete directory zstd: exit status 1 pg_restore: error: could not decompress data: compressed stream is incomplete custom lz4: exit status 1 pg_restore: error: could not read from input file: Input/output error directory lz4: exit status 1 pg_restore: error: could not uncompress data: (null) custom gzip: exit status 1 pg_restore: error: could not close data file "/tmp/pgdump-trunc.XjUiJ4/gzip-dir-bad/3931.dat": Undefined error: 0 directory gzip: exit status 1 ``` While testing, I also found a small issue in LZ4Stream_read_internal(). Its error branches call pg_log_error() and then return -1, but callers immediately call pg_fatal() when the return value <0. This results in duplicate error messages. So, I removed those pg_log_error() calls. See 0002 for the fix. I added tests only for zstd and lz4, since gzip is not changed. Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/
nocfbot_test_pgdump_comp.sh
Description: Binary data
v8-0001-Fix-detection-of-truncated-compressed-backups.patch
Description: Binary data
v8-0002-Fix-detection-of-truncated-zstd-and-LZ4-dump-data.patch
Description: Binary data
