Re: makewhatis on /usr
* Pascal Stumpf pascal.stu...@cubes.de [2011-11-20 01:25]: I think what Marc meant is that whatis.db/mandoc.db should be in the same directory as their corresponding manpages. i think you're reading that into his words, his point (rightly) was a file per dir (wherever that file actually lives) vs one global file. And I agree on that point, especially in the case of NFS-mounted /usr: One shouldn't have to run mandocdb to be able to run apropos(1) for the pages on the remote machine. And even worse, what happens if the remote machine is updated or packages are added? The client in such a setup will have an outdated database without being aware of it. given that these files are rebuilt periodically i think that is the much smaller drawback compared to writing to /usr. i've run into this before several times on CF-based systems with ro /usr, and this is the only offender i ever noticed. mandoc.db should be in the same directory as the manpages it was generated from, and need not be writable by machines that don't have write access to the pages themselves. i think you are overly optimistic on these. the files get rebuilt periodically for a reason - if they were reliably updated every time an new manpage shows up somewhere in the configured pathes that wouldn't be needed. admittedly these days with everybody using packages and pkg_add doing the right thing (afair at least), and people building from src beng dismissed anyway :), we're much closer to that optimum than ever before. With respect to the weekly makewhatis, I think that's a bug in the weekly(8) script: It should not blindly assume that every database listed in man.conf(5) is on a writable filesystem. that I agree with completely. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services, http://bsws.de, Full-Service ISP Secure Hosting, Mail and DNS Services. Dedicated Servers, Root to Fully Managed Henning Brauer Consulting, http://henningbrauer.com/
makewhatis on /usr
Hi, From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? From hier(7) : /usr/ Contains the majority of user utilities and applications. share/Architecture independent data files. man/ Manual pages. Is it really a manual page ?
Re: makewhatis on /usr
Thomas de Grivel tho...@lowh.net writes: Hi, From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? From hier(7) : /usr/ Contains the majority of user utilities and applications. share/Architecture independent data files. man/ Manual pages. Is it really a manual page ? Shouldn't the index of manual pages be with the manual pages? -- http://www.wistly.net
Re: makewhatis on /usr
On 11/19/11 15:40, Carson Chittom wrote: Thomas de Griveltho...@lowh.net writes: Hi, From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? From hier(7) : /usr/ Contains the majority of user utilities and applications. share/Architecture independent data files. man/ Manual pages. Is it really a manual page ? Shouldn't the index of manual pages be with the manual pages? Yeah and let's put the reader there too ?
Re: makewhatis on /usr
On Sat, Nov 19, 2011, Carson Chittom wrote: Thomas de Grivel tho...@lowh.net writes: Hi, From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? From hier(7) : /usr/ Contains the majority of user utilities and applications. share/Architecture independent data files. man/ Manual pages. Is it really a manual page ? Shouldn't the index of manual pages be with the manual pages? Should /etc/passwd be stored in /home instead? I think a readonly (or NFS shared) /usr is supposed to work, so anything that's regularly written should be in /var. That's what /var is for, after all.
Re: makewhatis on /usr
Hi Thomas, Thomas de Grivel wrote on Sat, Nov 19, 2011 at 03:32:03PM +0100: From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? I think you have a point. Specifically, /var/db/man/. Having cron(8) scripts write to /usr is ugly. I don't see why mounting /usr readonly should require disabling makewhatis(8). Right now, Kristaps and myself are rewriting makewhatis(8) and apropos(1) anyway, the databases will be renamed from whatis.db to mandoc.db and mandoc.index, and their format will be completely different, using db(3) instead of plain text. That might be a good time to move variable content out of /usr. At the apropriate time, i'll bring this up among developers. Please be patient for a few months, but remind me when you see a commit to src/etc/weekly or src/usr.sbin/pkg_add enabling mandocdb(8), but nothing happened regarding the placement of mandoc databases before that. Regarding the latter, watch out for commits to src/usr.bin/mandoc/mandocdb.h, where the paths used by mandocdb(8) and apropos(1) are defined; or, in case we redesign that, to src/usr.bin/mandoc/mandocdb.c and src/usr.bin/mandoc/apropos_db.c, where these paths are used. I'm not going to move whatis.db files around at this point. They are close to retirement, anyway. Thanks for bringing this up, Ingo
Re: makewhatis on /usr
On Sat, Nov 19, 2011 at 04:31:57PM +0100, Ingo Schwarze wrote: Hi Thomas, Thomas de Grivel wrote on Sat, Nov 19, 2011 at 03:32:03PM +0100: From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? I think you have a point. Specifically, /var/db/man/. Having cron(8) scripts write to /usr is ugly. I don't see why mounting /usr readonly should require disabling makewhatis(8). I disagree: manpage directories are self-contained. If I add or remove a directory to my man configuration, it shouldn't require a rebuild of the database for other directories. Hence having a whatis.db per-man directory root. If you want to move those to /var/db/man or something, you'd better be prepared to have database file names that depend on the root directory being used.
Re: makewhatis on /usr
On Sat, Nov 19, 2011 at 10:14:44AM -0500, Ted Unangst wrote: On Sat, Nov 19, 2011, Carson Chittom wrote: Thomas de Grivel tho...@lowh.net writes: Hi, From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? From hier(7) : /usr/ Contains the majority of user utilities and applications. share/Architecture independent data files. man/ Manual pages. Is it really a manual page ? Shouldn't the index of manual pages be with the manual pages? Should /etc/passwd be stored in /home instead? I think a readonly (or NFS shared) /usr is supposed to work, so anything that's regularly written should be in /var. That's what /var is for, after all. Under normal circumstances, /usr/share/man/whatis.db is NOT written to: makewhatis(8) does not even try to write the file if it didn't change. That particular problem only triggers after you upgrade your system if /usr is mounted read-only. That's one reason it's done at the end of make build: once you're done, you can remount /usr read-only.
Re: makewhatis on /usr
Hi Marc, Marc Espie wrote on Sat, Nov 19, 2011 at 05:40:38PM +0100: On Sat, Nov 19, 2011 at 04:31:57PM +0100, Ingo Schwarze wrote: Thomas de Grivel wrote on Sat, Nov 19, 2011 at 03:32:03PM +0100: From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? I think you have a point. Specifically, /var/db/man/. Having cron(8) scripts write to /usr is ugly. I don't see why mounting /usr readonly should require disabling makewhatis(8). I disagree: manpage directories are self-contained. If I add or remove a directory to my man configuration, it shouldn't require a rebuild of the database for other directories. Hence having a whatis.db per-man directory root. If you want to move those to /var/db/man or something, you'd better be prepared to have database file names that depend on the root directory being used. I fully agree with all that, and having per-hierarchy databases in /var/db/man - one for /usr/share/man, one for /usr/X11R6/man, one for /usr/local/man, one for each additional directory the user configures in man.conf(5) - is indeed what i hope to do. Yours, Ingo
Re: makewhatis on /usr
On Sat, 19 Nov 2011 18:05:42 +0100, Ingo Schwarze wrote: Hi Marc, Marc Espie wrote on Sat, Nov 19, 2011 at 05:40:38PM +0100: On Sat, Nov 19, 2011 at 04:31:57PM +0100, Ingo Schwarze wrote: Thomas de Grivel wrote on Sat, Nov 19, 2011 at 03:32:03PM +0100: From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? I think you have a point. Specifically, /var/db/man/. Having cron(8) scripts write to /usr is ugly. I don't see why mounting /usr readonly should require disabling makewhatis(8). I disagree: manpage directories are self-contained. If I add or remove a directory to my man configuration, it shouldn't require a rebuild of the database for other directories. Hence having a whatis.db per-man directory root. If you want to move those to /var/db/man or something, you'd better be prepared to have database file names that depend on the root directory being used. I fully agree with all that, and having per-hierarchy databases in /var/db/man - one for /usr/share/man, one for /usr/X11R6/man, one for /usr/local/man, one for each additional directory the user configures in man.conf(5) - is indeed what i hope to do. I think what Marc meant is that whatis.db/mandoc.db should be in the same directory as their corresponding manpages. And I agree on that point, especially in the case of NFS-mounted /usr: One shouldn't have to run mandocdb to be able to run apropos(1) for the pages on the remote machine. And even worse, what happens if the remote machine is updated or packages are added? The client in such a setup will have an outdated database without being aware of it. mandoc.db should be in the same directory as the manpages it was generated from, and need not be writable by machines that don't have write access to the pages themselves. With respect to the weekly makewhatis, I think that's a bug in the weekly(8) script: It should not blindly assume that every database listed in man.conf(5) is on a writable filesystem.
Re: makewhatis on /usr
Hi Pascal, Pascal Stumpf wrote on Sun, Nov 20, 2011 at 01:18:27AM +0100: On Sat, 19 Nov 2011 18:05:42 +0100, Ingo Schwarze wrote: Marc Espie wrote on Sat, Nov 19, 2011 at 05:40:38PM +0100: On Sat, Nov 19, 2011 at 04:31:57PM +0100, Ingo Schwarze wrote: Thomas de Grivel wrote on Sat, Nov 19, 2011 at 03:32:03PM +0100: From weekly output : Rebuilding whatis databases: /usr/libexec/makewhatis: Can't create /usr/share/man/whatis.db: Read-only file system Should not whatis.db be in /var/... ? I think you have a point. Specifically, /var/db/man/. Having cron(8) scripts write to /usr is ugly. I don't see why mounting /usr readonly should require disabling makewhatis(8). I disagree: manpage directories are self-contained. If I add or remove a directory to my man configuration, it shouldn't require a rebuild of the database for other directories. Hence having a whatis.db per-man directory root. If you want to move those to /var/db/man or something, you'd better be prepared to have database file names that depend on the root directory being used. I fully agree with all that, and having per-hierarchy databases in /var/db/man - one for /usr/share/man, one for /usr/X11R6/man, one for /usr/local/man, one for each additional directory the user configures in man.conf(5) - is indeed what i hope to do. I think what Marc meant is that whatis.db/mandoc.db should be in the same directory as their corresponding manpages. And I agree on that point, especially in the case of NFS-mounted /usr: One shouldn't have to run mandocdb to be able to run apropos(1) for the pages on the remote machine. And even worse, what happens if the remote machine is updated or packages are added? The client in such a setup will have an outdated database without being aware of it. mandoc.db should be in the same directory as the manpages it was generated from, and need not be writable by machines that don't have write access to the pages themselves. Hmm, probably you are right: The mandoc database only needs to be changed when manual pages are added to or removed from the tree, so /FOO/man/mandoc.db is _not_ more volatile than /FOO/man/man*/* and /FOO/bin/* themselves. With respect to the weekly makewhatis, I think that's a bug in the weekly(8) script: It should not blindly assume that every database listed in man.conf(5) is on a writable filesystem. Since weekly(8) just calls makewhatis(8) without any arguments, it can't do much about the fact that some directories might be locally mounted and writeable, while others may be on NFS and read-only. Besides, calling /usr/libexec/makewhatis by hand has to deal with those issues as well. Thus, the right solution probably is to have makewhatis(8) - or in the future, mandocdb(8) -, when running without arguments, test whether each database is writeable, before scanning the associated directory, and in case it is not, just skip the directory. On the other hand, when a directory is explicitly specified on the command line, the current behaviour is probably optimal: Build the database, if it did not change, be happy, otherwise, try to move the new one into its place, and if that fails, complain. Yours, Ingo
