[Kernel-packages] [Bug 1649342] Re: md5sum fails with message "Invalid argument" on 4, 294, 967, 295-byte files in FAT32 - tracked down to Ubuntu incompatible change to kernel's fread and stdio stream
I tested with Ubuntu with the latest updates and I cannot reproduce the
bug.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1649342
Title:
md5sum fails with message "Invalid argument" on 4,294,967,295-byte
files in FAT32 - tracked down to Ubuntu incompatible change to
kernel's fread and stdio stream
Status in linux package in Ubuntu:
Triaged
Status in linux source package in Yakkety:
Triaged
Status in linux source package in Zesty:
Triaged
Bug description:
Bug discovered in the threads at http://lists.gnu.org/archive/html
/bug-coreutils/2016-12/msg8.html and
https://bugzilla.kernel.org/show_bug.cgi?id=189981 partially quoted
below (with the permission of the persons involved).
=== Message 1 ===
--- Bug ---
In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one
byte less than the maximum file size) it succeeds, but if one runs md5sum on a
4,294,967,295-byte file (the maximum file size) it fails with error message
"Invalid argument".
--- How to reproduce the bug ---
Create a FAT32 file system in a file "tmp.fs":
truncate -s 9G tmp.fs
mkfs.vfat -F 32 tmp.fs
Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
sudo mkdir /tmp/mounted_tmp/
sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294
bytes and file "file_2" with 4,294,967,295 bytes:
cd /tmp/mounted_tmp/
truncate -s 4294967294 file_1
truncate -s 4294967295 file_2
Run md5sum on the two files "file_1" and "file_2":
md5sum file_1
md5sum file_2
The outputs should be respectively (notice that the second output is an error
message):
541249e3205af07b4a03f891185f64a0 file_1
md5sum: file_2: Invalid argument
Unmount the file system at "/tmp/mounted_tmp/":
cd ..
sudo umount /tmp/mounted_tmp/
sudo rmdir /tmp/mounted_tmp/
Remove the file "tmp.fs".
--- Notes ---
Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel
4.8.0-30-generic.
The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum,
but not crc32.
=== Message 2 ===
...
> --- How to reproduce the bug ---
...
I can't repro this with any md5sum version on 4.2.5-300.fc23.x86_64
So I'm guessing a kernel regression.
Can you strace -o /tmp/md5sum.strace md5sum file_2,
and look towards the end of the strace file to identify the syscall returning
EINVAL?
In any case I'd direct the issue towards the kernel folks.
...
=== Message 3 ===
> Can you strace -o /tmp/md5sum.strace md5sum file_2,
> and look towards the end of the strace file to identify the syscall
> returning EINVAL?
It is the seventh and eighth lines below:
...
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32767
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
write(2, "md5sum: ", 8) = 8
write(2, "file_2", 6) = 6
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
read(4, "", 4096) = 0
close(4)= 0
open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f53d8d0a000
close(4)= 0
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, ": Invalid argument", 18) = 18
write(2, "\n", 1) = 1
lseek(3, 0, SEEK_CUR) = 4294967295
close(3)= 0
close(1)= 0
close(2)= 0
exit_group(1) = ?
+++ exited with 1 +++
=== Message 4 ===
...
>read(3,
[Kernel-packages] [Bug 1649342] Re: md5sum fails with message "Invalid argument" on 4, 294, 967, 295-byte files in FAT32 - tracked down to Ubuntu incompatible change to kernel's fread and stdio stream
I tested with kernel 4.10.041000-rc5-generic from
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.10-rc5/ and I cannot
reproduce the bug.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1649342
Title:
md5sum fails with message "Invalid argument" on 4,294,967,295-byte
files in FAT32 - tracked down to Ubuntu incompatible change to
kernel's fread and stdio stream
Status in linux package in Ubuntu:
Triaged
Status in linux source package in Yakkety:
Triaged
Status in linux source package in Zesty:
Triaged
Bug description:
Bug discovered in the threads at http://lists.gnu.org/archive/html
/bug-coreutils/2016-12/msg8.html and
https://bugzilla.kernel.org/show_bug.cgi?id=189981 partially quoted
below (with the permission of the persons involved).
=== Message 1 ===
--- Bug ---
In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one
byte less than the maximum file size) it succeeds, but if one runs md5sum on a
4,294,967,295-byte file (the maximum file size) it fails with error message
"Invalid argument".
--- How to reproduce the bug ---
Create a FAT32 file system in a file "tmp.fs":
truncate -s 9G tmp.fs
mkfs.vfat -F 32 tmp.fs
Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
sudo mkdir /tmp/mounted_tmp/
sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294
bytes and file "file_2" with 4,294,967,295 bytes:
cd /tmp/mounted_tmp/
truncate -s 4294967294 file_1
truncate -s 4294967295 file_2
Run md5sum on the two files "file_1" and "file_2":
md5sum file_1
md5sum file_2
The outputs should be respectively (notice that the second output is an error
message):
541249e3205af07b4a03f891185f64a0 file_1
md5sum: file_2: Invalid argument
Unmount the file system at "/tmp/mounted_tmp/":
cd ..
sudo umount /tmp/mounted_tmp/
sudo rmdir /tmp/mounted_tmp/
Remove the file "tmp.fs".
--- Notes ---
Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel
4.8.0-30-generic.
The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum,
but not crc32.
=== Message 2 ===
...
> --- How to reproduce the bug ---
...
I can't repro this with any md5sum version on 4.2.5-300.fc23.x86_64
So I'm guessing a kernel regression.
Can you strace -o /tmp/md5sum.strace md5sum file_2,
and look towards the end of the strace file to identify the syscall returning
EINVAL?
In any case I'd direct the issue towards the kernel folks.
...
=== Message 3 ===
> Can you strace -o /tmp/md5sum.strace md5sum file_2,
> and look towards the end of the strace file to identify the syscall
> returning EINVAL?
It is the seventh and eighth lines below:
...
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32767
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
write(2, "md5sum: ", 8) = 8
write(2, "file_2", 6) = 6
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
read(4, "", 4096) = 0
close(4)= 0
open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f53d8d0a000
close(4)= 0
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, ": Invalid argument", 18) = 18
write(2, "\n", 1) = 1
lseek(3, 0, SEEK_CUR) = 4294967295
close(3)= 0
close(1)= 0
close(2)= 0
exit_group(1) = ?
+++ exited with 1 +++
===
[Kernel-packages] [Bug 1649342] Re: md5sum fails with message "Invalid argument" on 4, 294, 967, 295-byte files in FAT32 - tracked down to Ubuntu incompatible change to kernel's fread and stdio stream
I tested with kernel 4.8.0-30-generic from
http://kernel.ubuntu.com/~jsalisbury/lp1649342/ and I cannot reproduce
the bug.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1649342
Title:
md5sum fails with message "Invalid argument" on 4,294,967,295-byte
files in FAT32 - tracked down to Ubuntu incompatible change to
kernel's fread and stdio stream
Status in linux package in Ubuntu:
Triaged
Bug description:
Bug discovered in the threads at http://lists.gnu.org/archive/html
/bug-coreutils/2016-12/msg8.html and
https://bugzilla.kernel.org/show_bug.cgi?id=189981 partially quoted
below (with the permission of the persons involved).
=== Message 1 ===
--- Bug ---
In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one
byte less than the maximum file size) it succeeds, but if one runs md5sum on a
4,294,967,295-byte file (the maximum file size) it fails with error message
"Invalid argument".
--- How to reproduce the bug ---
Create a FAT32 file system in a file "tmp.fs":
truncate -s 9G tmp.fs
mkfs.vfat -F 32 tmp.fs
Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
sudo mkdir /tmp/mounted_tmp/
sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294
bytes and file "file_2" with 4,294,967,295 bytes:
cd /tmp/mounted_tmp/
truncate -s 4294967294 file_1
truncate -s 4294967295 file_2
Run md5sum on the two files "file_1" and "file_2":
md5sum file_1
md5sum file_2
The outputs should be respectively (notice that the second output is an error
message):
541249e3205af07b4a03f891185f64a0 file_1
md5sum: file_2: Invalid argument
Unmount the file system at "/tmp/mounted_tmp/":
cd ..
sudo umount /tmp/mounted_tmp/
sudo rmdir /tmp/mounted_tmp/
Remove the file "tmp.fs".
--- Notes ---
Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel
4.8.0-30-generic.
The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum,
but not crc32.
=== Message 2 ===
...
> --- How to reproduce the bug ---
...
I can't repro this with any md5sum version on 4.2.5-300.fc23.x86_64
So I'm guessing a kernel regression.
Can you strace -o /tmp/md5sum.strace md5sum file_2,
and look towards the end of the strace file to identify the syscall returning
EINVAL?
In any case I'd direct the issue towards the kernel folks.
...
=== Message 3 ===
> Can you strace -o /tmp/md5sum.strace md5sum file_2,
> and look towards the end of the strace file to identify the syscall
> returning EINVAL?
It is the seventh and eighth lines below:
...
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32767
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
write(2, "md5sum: ", 8) = 8
write(2, "file_2", 6) = 6
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
read(4, "", 4096) = 0
close(4)= 0
open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f53d8d0a000
close(4)= 0
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, ": Invalid argument", 18) = 18
write(2, "\n", 1) = 1
lseek(3, 0, SEEK_CUR) = 4294967295
close(3)= 0
close(1)= 0
close(2)= 0
exit_group(1) = ?
+++ exited with 1 +++
=== Message 4 ===
...
>read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32767
[Kernel-packages] [Bug 1649342] Re: md5sum fails with message "Invalid argument" on 4, 294, 967, 295-byte files in FAT32 - tracked down to Ubuntu incompatible change to kernel's fread and stdio stream
I tested with kernel 4.8.4-040804-generic from
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.8.4/ and I can reproduce the
bug.
I tested also with kernel 4.9.0-040900-generic from
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.9/ and I can also reproduce
the bug.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1649342
Title:
md5sum fails with message "Invalid argument" on 4,294,967,295-byte
files in FAT32 - tracked down to Ubuntu incompatible change to
kernel's fread and stdio stream
Status in linux package in Ubuntu:
Triaged
Bug description:
Bug discovered in the threads at http://lists.gnu.org/archive/html
/bug-coreutils/2016-12/msg8.html and
https://bugzilla.kernel.org/show_bug.cgi?id=189981 partially quoted
below (with the permission of the persons involved).
=== Message 1 ===
--- Bug ---
In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one
byte less than the maximum file size) it succeeds, but if one runs md5sum on a
4,294,967,295-byte file (the maximum file size) it fails with error message
"Invalid argument".
--- How to reproduce the bug ---
Create a FAT32 file system in a file "tmp.fs":
truncate -s 9G tmp.fs
mkfs.vfat -F 32 tmp.fs
Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
sudo mkdir /tmp/mounted_tmp/
sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294
bytes and file "file_2" with 4,294,967,295 bytes:
cd /tmp/mounted_tmp/
truncate -s 4294967294 file_1
truncate -s 4294967295 file_2
Run md5sum on the two files "file_1" and "file_2":
md5sum file_1
md5sum file_2
The outputs should be respectively (notice that the second output is an error
message):
541249e3205af07b4a03f891185f64a0 file_1
md5sum: file_2: Invalid argument
Unmount the file system at "/tmp/mounted_tmp/":
cd ..
sudo umount /tmp/mounted_tmp/
sudo rmdir /tmp/mounted_tmp/
Remove the file "tmp.fs".
--- Notes ---
Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel
4.8.0-30-generic.
The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum,
but not crc32.
=== Message 2 ===
...
> --- How to reproduce the bug ---
...
I can't repro this with any md5sum version on 4.2.5-300.fc23.x86_64
So I'm guessing a kernel regression.
Can you strace -o /tmp/md5sum.strace md5sum file_2,
and look towards the end of the strace file to identify the syscall returning
EINVAL?
In any case I'd direct the issue towards the kernel folks.
...
=== Message 3 ===
> Can you strace -o /tmp/md5sum.strace md5sum file_2,
> and look towards the end of the strace file to identify the syscall
> returning EINVAL?
It is the seventh and eighth lines below:
...
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32768
read(3,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) =
32767
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
read(3, 0x25c12b0, 8192)= -1 EINVAL (Invalid argument)
write(2, "md5sum: ", 8) = 8
write(2, "file_2", 6) = 6
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
read(4, "", 4096) = 0
close(4)= 0
open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
(No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No
such file or directory)
open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f53d8d0a000
close(4)= 0
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1
ENOENT (No such file or directory)
write(2, ": Invalid argument", 18) = 18
write(2, "\n", 1) = 1
lseek(3, 0, SEEK_CUR) = 4294967295
close(3)= 0
close(1)= 0
close(2)= 0
exit_group(1) = ?
+++
