I killed my FreeBSD 4.8-PRE (2003-02-18) server today by exhausting the
available mbufs.  I'd seen warnings like "All mbuf clusters exhausted,
please see tuning(7)." in /var/log/messages, so I added
`kern.ipc.nmbclusters="16384"' to /boot/loader.conf.local a while back.

This server has 768MB of memory, and I run at least two copies of PostgreSQL
(it has multiple jails) at any given time and use it to server NFS to a
couple of Linux clients.  Today I wanted to test Bochs on one of the Linux
machines, ran the bochsconf program, watched it run "createdisk" to make a
512MB disk image in my home directory, and saw the server panic within a few
seconds.  When it came back up, I bumped nmbclusters to 32768, rebooted, and
tried createdisk it.  This time I watched `netstat -m' as the dreaded "All
mbuf clusters exhausted..." message began to scroll up the screen.  This
time, the server stayed alive, but it clearly maxed out all available
buffers.  After createdisk was finished, the mbuf usage went back to normal:

    $ netstat -m
    389/36736/131072 mbufs in use (current/peak/max):
            389 mbufs allocated to data
    388/32768/32768 mbuf clusters in use (current/peak/max)
    74720 Kbytes allocated to network (33% of mb_map in use)
    345 requests for memory denied
    1 requests for memory delayed
    0 calls to protocol drain routines

My questions are these:

1) Is there a guideline for how many nmbclusters to allocate?  Short of
   running out of physical memory, is there such a thing as too many?

2) I've already allocated 64MB to network buffering.  This seems like a huge
   amount to me.  Is it?

3) Should running out of mbufs be expected to crash the server, or should I
   try to reproduce it and file a PR?

4) What could createdisk have been doing to spike usage that heavily?

5) Could Linux's NFS client implementation have aggravated the situation?

-- 
Kirk Strauser
In Googlis non est, ergo non est.

Attachment: pgp00000.pgp
Description: PGP signature

Reply via email to