The db hash table cache size used by pwd_mkdb is currently limited
to 12MB. For very large passwd files this results in a lot of extra
disk I/O. Increasing the max to 32MB makes pwd_mkdb several times
faster on a real-world master.passwd files that is 264829 lines.
On an amd64, the resident size of pwd_mkdb is 35MB with the 12MB
cache limit and 73MB with a 32MB cache limit. Not every machine
has gigabytes of RAM but low-memory machines are not likely to be
building enormous passwd databases.
While here, bump the bucket size to match the optimal I/O blocksize
for the filesystem if it is larger than 4096. The default db
behavior is to use st_blksize if no bsize is specified so we could
just set bsize to 0 instead.
Opinions? NetBSD has an option for pwd_mkdb to set the cache size
on the command line but I'd rather just have better default behavior.
- todd
Index: usr.sbin/pwd_mkdb/pwd_mkdb.c
===================================================================
RCS file: /cvs/src/usr.sbin/pwd_mkdb/pwd_mkdb.c,v
retrieving revision 1.53
diff -u -p -u -r1.53 pwd_mkdb.c
--- usr.sbin/pwd_mkdb/pwd_mkdb.c 5 Nov 2015 15:10:11 -0000 1.53
+++ usr.sbin/pwd_mkdb/pwd_mkdb.c 11 Sep 2018 20:16:51 -0000
@@ -187,9 +187,12 @@ main(int argc, char **argv)
/* Tweak openinfo values for large passwd files. */
if (st.st_size > (off_t)100*1024)
- openinfo.cachesize = MINIMUM(st.st_size * 20,
(off_t)12*1024*1024);
+ openinfo.cachesize = MINIMUM(st.st_size * 20,
(off_t)32*1024*1024);
if (st.st_size / 128 > openinfo.nelem)
openinfo.nelem = st.st_size / 128;
+
+ if (st.st_blksize > openinfo.bsize)
+ openinfo.bsize = st.st_blksize;
/* If only updating a single record, stash the old uid */
if (username) {
