The dm_test_cmd_zip_gzwrite sandbox test occasionally fails in CI
with:
12582912/16777216
Error: inflate() returned -5
The chunked decompression loop added in commit 58e523fedf48 ("gunzip:
Implement chunked decompression") treats Z_BUF_ERROR from inflate()
as fatal. When an input chunk is exhausted at exactly the same time
as the write buffer fills up, the next inflate() call is made with
avail_in == 0, cannot make progress, and returns Z_BUF_ERROR. Per the
zlib documentation this only means "no progress was possible" and the
call should be repeated with more input, which is what the reference
implementation in zlib examples/zpipe.c does.
The failure needs the consumed/produced byte counts to line up with
both the chunk size and the write buffer size at once, with no
buffered output on the inflate side, which is why only certain random
payloads trigger it. Note that the failure offset above is a multiple
of the 1 MiB write buffer while gzwrite_chunk was SZ_1M + 1.
Patch 1 makes gzwrite() refill the input chunk in this situation.
Patch 2 adds a deterministic regression test which builds a gzip file
from two stored deflate blocks by hand and aligns the chunk boundary
with the write buffer boundary exactly, failing reliably without
patch 1.
Verified on sandbox and sandbox64:
- dm_test_cmd_gzwrite_chunk_boundary fails with -5 in 20 out of 20
runs before the fix, passes 100 out of 100 runs after
- dm_test_cmd_zip_gzwrite fails 17 out of 2000 runs (about 1%)
before the fix, every time with the same signature as the CI
flake, and passes 2000 out of 2000 runs after
- dm_test_cmd_zip_unzip keeps passing
Aristo Chen (2):
gunzip: Fix spurious Z_BUF_ERROR in chunked gzwrite decompression
test: cmd: Add gzwrite chunk boundary regression test
lib/gunzip.c | 10 +++++
test/cmd/unzip.c | 103 ++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 111 insertions(+), 2 deletions(-)
--
2.43.0
