i spent the last 4 days searching because of this bug.
originally i started to write a bug report against libgdk-pixbuf2.0-0
but further investigations revealed the ugly root cause.
it was very hard to "google" this. maybe adding some pointers and
related keywords here will help upcoming victims...
as enrico reported, update-mime-database will not check readdir() for
error-return. instead it will silently create an emtpy mime cache.
this will cause e.g. all gdk pixbuf users to get an error message like
Couldn’t recognize the image file format for file XY
and that while
/usr/lib/ARCH/gdk-pixbuf-2.0/2.10.0/loaders.cache
is completely fine.
gdk pixbuf relies on glib's xdg_mime stuff which reads
/usr/share/mime/mime.cache
it might sound like an unimportant bug, to fix that error checking, but
it seems the only good way to notify people that there is something
wrong going on.
and i think it definitely is a bug within update-mime-database to
silently and happily create an empty mime cache while there always is at
least freedesktop.org.xml to be expected in /usr/share/mime/packages/
because it is installed with that package.
so i would suggest to at least output some warning message in case
/usr/share/mime/packages/ is discovered completely empty.
background:
running 32bit processes via qemu-user-static on a 64bit linux kernel
with a chroot will cause readdir() to error-out with
EOVERFLOW - Value too large for defined data type
for processes that are compiled without "-D_FILE_OFFSET_BITS=64" and for
directories on an ext4 filesystem.
this problem surfaced because of a change in glibc:
https://sourceware.org/bugzilla/show_bug.cgi?id=23960
qemu bug:
https://bugs.launchpad.net/qemu/+bug/1805913
linux kernel bug:
https://bugzilla.kernel.org/show_bug.cgi?id=205957
lkml:
https://lkml.org/lkml/2018/12/27/155
i stumbled upon this when preparing an armhf buster rootfs via
multistrap (or debootstrap) and qemu-user-static on my x86_64 buster
host that uses an ext4 filesystem.
if update-mime-database would error-out because of that readdir() error
i would have seen that something is wrong and had something to search for.
with the current situation i had no indication at all and had to start
debugging gdk pixbuf, leading to glib, leading to an empty mime.cache
and further debugging of update-mime-database leading to the EOVERFLOW
error.
please add this error-out to save others from having to go through this!
a workaround that made readdir() work in that scenario was to compile it
with large-file support (-D_FILE_OFFSET_BITS=64) which can be achieved
by adding the macro
AC_SYS_LARGEFILE
to configure.ac.
attached a patch that does both.
--
Florian Schmidt
diff -Nurw shared-mime-info-1.10/configure.ac shared-mime-info-1.10-readdir-error-check/configure.ac
--- shared-mime-info-1.10/configure.ac 2018-06-28 10:02:17.000000000 +0000
+++ shared-mime-info-1.10-readdir-error-check/configure.ac 2020-03-03 20:49:32.000000000 +0000
@@ -13,6 +13,7 @@
dnl Checks for typedefs, structures, and compiler characteristics.
AC_C_CONST
+AC_SYS_LARGEFILE
IT_PROG_INTLTOOL([0.35.0])
diff -Nurw shared-mime-info-1.10/update-mime-database.c shared-mime-info-1.10-readdir-error-check/update-mime-database.c
--- shared-mime-info-1.10/update-mime-database.c 2018-04-24 13:09:19.000000000 +0000
+++ shared-mime-info-1.10-readdir-error-check/update-mime-database.c 2020-03-03 20:50:32.000000000 +0000
@@ -804,9 +804,12 @@
}
files = g_ptr_array_new();
- while ((ent = readdir(dir)))
+ while (TRUE)
{
int l;
+ errno = 0;
+ if(!(ent = readdir(dir)))
+ break;
l = strlen(ent->d_name);
if (l < 4 || strcmp(ent->d_name + l - 4, ".xml") != 0)
continue;
@@ -817,6 +820,11 @@
}
g_ptr_array_add(files, g_strdup(ent->d_name));
}
+ if (errno != 0)
+ {
+ g_printerr("readdir for %s returned errno %d: %s\n", path, errno, strerror(errno));
+ exit(EXIT_FAILURE);
+ }
closedir(dir);
g_ptr_array_sort(files, strcmp2);
