Package: moreutils Version: 0.67-1 Severity: normal Tags: patch, upstream On macOS (Darwin 64-bit), `sponge` fails when reading files of size ≥2 GiB:
$ truncate -s 2147483648 in
$ cat in | sponge out
sponge: failed to read from stdin: Invalid argument
This happens because Darwin's `read(2)` system call rejects requests
where `count > INT_MAX` (2,147,483,647), even though SSIZE_MAX is much
larger on 64-bit. When sponge tries to read 2 GiB in a single call,
`read()` fails with EINVAL.
Linux accepts such large reads, so the issue only shows up on macOS and
BSDs that impose the INT_MAX limit.
Expected behavior:
`sponge` should be able to handle arbitrarily large inputs, regardless
of platform, by reading in chunks up to the maximum allowed size.
Proposed fix:
Limit the size of each `read()` call to `min(SSIZE_MAX, INT_MAX)`.
I’ve written a patch which adds a small `safe_read()` helper and uses it
in the two places where `sponge` reads input.
Patch is attached:
moreutils-e615f47-sponge: Fix "Invalid argument" error with 2GB+ files on
macOS.patch
This has been tested on macOS Sonoma with 2–3 GiB files, and works
correctly. The performance impact is negligible (<1 extra syscall per
2 GiB read).
Thanks for maintaining moreutils!
Cornelius Roemer
bin6CSe2i2LtM.bin
Description: Binary data
