Public bug reported:
Bug discovered in the threads at http://lists.gnu.org/archive/html/bug-
coreutils/2016-12/msg00008.html and
https://bugzilla.kernel.org/show_bug.cgi?id=189981 partially quoted
below (with the permission of the persons involved).
=== Message 1 ===
--- Bug ---
In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one
byte less than the maximum file size) it succeeds, but if one runs md5sum on a
4,294,967,295-byte file (the maximum file size) it fails with error message
"Invalid argument".
--- How to reproduce the bug ---
Create a FAT32 file system in a file "tmp.fs":
truncate -s 9G tmp.fs
mkfs.vfat -F 32 tmp.fs
Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
sudo mkdir /tmp/mounted_tmp/
sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294 bytes
and file "file_2" with 4,294,967,295 bytes:
cd /tmp/mounted_tmp/
truncate -s 4294967294 file_1
truncate -s 4294967295 file_2
Run md5sum on the two files "file_1" and "file_2":
md5sum file_1
md5sum file_2
The outputs should be respectively (notice that the second output is an error
message):
541249e3205af07b4a03f891185f64a0 file_1
md5sum: file_2: Invalid argument
Unmount the file system at "/tmp/mounted_tmp/":
cd ..
sudo umount /tmp/mounted_tmp/
sudo rmdir /tmp/mounted_tmp/
Remove the file "tmp.fs".
--- Notes ---
Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel
4.8.0-30-generic.
The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum,
but not crc32.
=== Message 2 ===
...
> --- How to reproduce the bug ---
...
I can't repro this with any md5sum version on 4.2.5-300.fc23.x86_64
So I'm guessing a kernel regression.
Can you strace -o /tmp/md5sum.strace md5sum file_2,
and look towards the end of the strace file to identify the syscall returning
EINVAL?
In any case I'd direct the issue towards the kernel folks.
...
=== Message 3 ===
> Can you strace -o /tmp/md5sum.strace md5sum file_2,
> and look towards the end of the strace file to identify the syscall
> returning EINVAL?
It is the seventh and eighth lines below:
...
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32767
read(3, 0x25c12b0, 8192) = -1 EINVAL (Invalid argument)
read(3, 0x25c12b0, 8192) = -1 EINVAL (Invalid argument)
write(2, "md5sum: ", 8) = 8
write(2, "file_2", 6) = 6
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
read(4, "", 4096) = 0
close(4) = 0
open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f53d8d0a000
close(4) = 0
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, ": Invalid argument", 18) = 18
write(2, "\n", 1) = 1
lseek(3, 0, SEEK_CUR) = 4294967295
close(3) = 0
close(1) = 0
close(2) = 0
exit_group(1) = ?
+++ exited with 1 +++
=== Message 4 ===
...
> read(3,
> "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768)
> = 32767
OK we've read all we can, but to verify md5sum will do:
fread(buffer + 32767, 1, 1, stream)
Then the stdio stream will issue the underlying read()s
I'm not too sure where there are two reads here,
but they shouldn't be returning EINVAL, but just
returning 0 to indicate EOF.
> read(3, 0x25c12b0, 8192) = -1 EINVAL (Invalid argument)
> read(3, 0x25c12b0, 8192) = -1 EINVAL (Invalid argument)
So it's a kernel bug as suspected.
...
=== Message 5 ===
...
Hm, tested on debian/testing with v4.8 vanilla (+ debian gcc PIE fix).
However, I couldn't reproduce it.
# truncate -s 9G tmp.fs
# mkfs.vfat -F 32 tmp.fs
mkfs.fat 4.0 (2016-05-06)
# mount -o loop,rw,uid=1000,gid=1000 tmp.fs m
# cd m
# truncate -s 4294967294 file_1
# truncate -s 4294967295 file_2
# md5sum file_1
541249e3205af07b4a03f891185f64a0 file_1
# md5sum file_2
c654ebc4b3472cfa01ade24bbbbc6d3e file_2
Can you try v4.8 vanilla?
...
=== Message 6 ===
...
I tested with kernel 4.8.0-040800-generic from
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.8/ and I cannot reproduce the
bug.
I tested with kernel 4.8.0-30-generic from an updated Ubuntu 16.10 and I can
reproduce the bug.
=== Message 7 ===
...
With quick check, ubuntu seems to added incompatible change. This should be
reported to ubuntu.
@@ -1674,6 +1687,10 @@
unsigned int prev_offset;
int error = 0;
+ if (unlikely(*ppos >= inode->i_sb->s_maxbytes))
+ return -EINVAL;
+ iov_iter_truncate(iter, inode->i_sb->s_maxbytes);
+
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1649342
Title:
md5sum fails with message "Invalid argument" on 4,294,967,295-byte
files in FAT32 - tracked down to Ubuntu incompatible change to
kernel's fread and stdio stream
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1649342/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
