Bug#1004083: plocate: stops with misleading error on pruned filesystem
On Sun, Jan 30, 2022 at 01:42:54PM -0700, Kevin Locke wrote: > Sure thing. After re-creating the broken CIFS mount on /mnt, I ran: > > updatedb -o /tmp/plocate.db > locate -d /tmp/plocate.db debian_version Great! Thanks for testing; I'll probably push out a new version shortly. /* Steinar */ -- Homepage: https://www.sesse.net/
Bug#1004083: plocate: stops with misleading error on pruned filesystem
On Sun, 2022-01-30 at 21:16 +0100, Steinar H. Gunderson wrote: > On Sun, Jan 30, 2022 at 07:25:55PM +0100, Steinar H. Gunderson wrote: >> Yes, I wonder if this is the best fix for this specific issue. >> If there's an error, there's no performance issue, since the alternative >> is total failure; we can check /proc/mounts before deciding to crash out >> or nt. > > I've pushed a change for this to upstream git; are you in a position to try > it out? Sure thing. After re-creating the broken CIFS mount on /mnt, I ran: updatedb -o /tmp/plocate.db locate -d /tmp/plocate.db debian_version I can confirm that with 5aea022 updatedb prints /mnt: Host is down then exits with code 1 (and does not create /tmp/plocate.db). With e734063 updatedb exits with code 0 and no error is printed. locate prints /etc/debian_version as expected. Those both seem like great improvements to me. Thank you for the quick response and fixes! Cheers, Kevin
Bug#1004083: plocate: stops with misleading error on pruned filesystem
On Sun, Jan 30, 2022 at 07:25:55PM +0100, Steinar H. Gunderson wrote: > Yes, I wonder if this is the best fix for this specific issue. > If there's an error, there's no performance issue, since the alternative > is total failure; we can check /proc/mounts before deciding to crash out > or nt. I've pushed a change for this to upstream git; are you in a position to try it out? /* Steinar */ -- Homepage: https://www.sesse.net/
Bug#1004083: plocate: stops with misleading error on pruned filesystem
On Sun, Jan 30, 2022 at 11:10:00AM -0700, Kevin Locke wrote: > I think the behavior is understandable in both of the above cases, > although it's unfortunate that a flakey mount on an excluded > filesystem at an otherwise always empty directory causes updatedb to > fail entirely. I'm not sure if it might be worth the effort to check > /proc/mounts or statfs(2) on stat(2) failure (for certain error > codes?), or to consider whether the directory is currently empty in > the database, or to skip it without discarding previous entries, or > something else. I'll defer to your judgement. Yes, I wonder if this is the best fix for this specific issue. If there's an error, there's no performance issue, since the alternative is total failure; we can check /proc/mounts before deciding to crash out or nt. /* Steinar */ -- Homepage: https://www.sesse.net/
Bug#1004083: plocate: stops with misleading error on pruned filesystem
Thanks for the thoughtful response! On Sun, 2022-01-30 at 18:47 +0100, Steinar H. Gunderson wrote: > On Thu, Jan 20, 2022 at 08:05:14AM -0700, Kevin Locke wrote: >> 2. The error occurred on a CIFS filesystem, which is excluded by >>PRUNEFS in /etc/updatedb.conf. Presumably it should be skipped >>rather than causing an error. > > The problem is that updatedb doesn't really know for sure it entered a new > filesystem until it's stat-ed the directory. [...] > >> 3. updatedb immediately aborts, rather than updating the database for >>the rest of the filesystem. In this case, since the failing mount >>was at the root, the entire tree is skipped. > > I don't honestly think this is a bug. If updatedb goes wrong for whatever > reason, it's not good to just silently discard parts of the database. > If nothing else, the next rescan could become very expensive. I think the behavior is understandable in both of the above cases, although it's unfortunate that a flakey mount on an excluded filesystem at an otherwise always empty directory causes updatedb to fail entirely. I'm not sure if it might be worth the effort to check /proc/mounts or statfs(2) on stat(2) failure (for certain error codes?), or to consider whether the directory is currently empty in the database, or to skip it without discarding previous entries, or something else. I'll defer to your judgement. It's not a common or major issue for me. Feel free to close or treat as low-priority. I thought it was worth raising the issue for consideration, but don't have strong feelings about it. Cheers, Kevin
Bug#1004083: plocate: stops with misleading error on pruned filesystem
On Thu, Jan 20, 2022 at 08:05:14AM -0700, Kevin Locke wrote: > It turns out this the failure was due to a network issue which > prevented contacting the host for the CIFS mount on /mnt. However, > this presents several issues that might be worth addressing: > > 1. The error message contains the parent directory, rather than the >directory which could not be read, which makes the error harder to >diagnose. This is a real bug, and should be easy to fix. > 2. The error occurred on a CIFS filesystem, which is excluded by >PRUNEFS in /etc/updatedb.conf. Presumably it should be skipped >rather than causing an error. The problem is that updatedb doesn't really know for sure it entered a new filesystem until it's stat-ed the directory. This is logic that's inherited from mlocate, and I guess it would be possible to rewrite it in theory, but such changes tend to be at great peril :-) So what's happening is basically that it stats the directory, checks if is device number is different from the parent, and if so, checks the filesystem from /proc/mounts. You can't just scan it once at the start and be done with it, because someone could be mounting new paths while you're doing updatedb (remember, updatedb can take hours), so you'd need a way to deal with new mounts appearing. (I already have complaints it doesn't immediately skip out if someone _unmounts_ something it's in the process of scanning!) The bind mount code has some special checks for this, since it cannot rely on st_dev (there's a separate thread waiting for changes to mountinfo), but it's fundamentally racy. > 3. updatedb immediately aborts, rather than updating the database for >the rest of the filesystem. In this case, since the failing mount >was at the root, the entire tree is skipped. I don't honestly think this is a bug. If updatedb goes wrong for whatever reason, it's not good to just silently discard parts of the database. If nothing else, the next rescan could become very expensive. /* Steinar */ -- Homepage: https://www.sesse.net/
Bug#1004083: plocate: stops with misleading error on pruned filesystem
Package: plocate Version: 1.1.14-1 Severity: normal Dear Maintainer, On a machine with a CIFS mount on /mnt, plocate-updatedb.service failed and updatedb.plocate printed: /: Host is down Running updatedb.plocate -v printed: //initrd.img.old //vmlinuz.old //mnt //root //vmlinuz //sys //bin //run //etc //libx32 //lib32 //var //initrd.img //usr //lib //tmp //proc //srv //home //dev //lost+found //lib64 //media //sbin //opt //boot /: Host is down It turns out this the failure was due to a network issue which prevented contacting the host for the CIFS mount on /mnt. However, this presents several issues that might be worth addressing: 1. The error message contains the parent directory, rather than the directory which could not be read, which makes the error harder to diagnose. 2. The error occurred on a CIFS filesystem, which is excluded by PRUNEFS in /etc/updatedb.conf. Presumably it should be skipped rather than causing an error. 3. updatedb immediately aborts, rather than updating the database for the rest of the filesystem. In this case, since the failing mount was at the root, the entire tree is skipped. Let me know if you'd like me to report any of these issues separately. Thanks for considering, Kevin -- System Information: Debian Release: bookworm/sid APT prefers testing-debug APT policy: (990, 'testing-debug'), (990, 'testing'), (500, 'unstable-debug'), (500, 'stable-debug'), (500, 'oldoldstable'), (500, 'unstable'), (101, 'experimental'), (1, 'experimental-debug') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.16.0+ (SMP w/4 CPU threads; PREEMPT) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages plocate depends on: ii libc6 2.33-2 ii libgcc-s1 11.2.0-13 ii libstdc++6 11.2.0-13 ii liburing2 2.1-2 ii libzstd11.4.8+dfsg-3 plocate recommends no packages. Versions of packages plocate suggests: ii powermgmt-base 1.36 ii systemd-sysv250.2-3 -- no debconf information
