On 26. feb. 2010, at 11.58, Claudio Jeker wrote: > On Fri, Feb 26, 2010 at 11:30:30AM +0100, Pete Vickers wrote: >> On 26. feb. 2010, at 03.01, Aaron Mason wrote: >> >>> On Thu, Feb 25, 2010 at 10:04 AM, Pete Vickers <p...@systemnet.no> wrote: >>>> Hi, >>>> >>>> A proxy (squid) server running i368/4.6RELEASE with around 800 users, what >>>> would be a reasonable value to increase kern.maxclusters too, to cure this >> : >>>> >>>> >>>> r...@proxy-s ~> grep mcl /var/log/messages >>>> Dec 10 10:13:43 proxy-s /bsd: WARNING: mclpools limit reached; increase >>>> kern.maxclusters >>>> Dec 10 11:06:07 proxy-s /bsd: WARNING: mclpools limit reached; increase >>>> kern.maxclusters >>>> Dec 15 13:41:48 proxy-s /bsd: WARNING: mclpools limit reached; increase >>>> kern.maxclusters >>>> >>>> >>>> r...@proxy-s ~> sysctl kern.maxclusters >>>> kern.maxclusters=6144 >>>> >>>> >>>> r...@proxy-s ~> netstat -m >>>> 4098 mbufs in use: >>>> 1131 mbufs allocated to data >>>> 2962 mbufs allocated to packet headers >>>> 5 mbufs allocated to socket names and addresses >>>> 1084/6152/6144 mbuf 2048 byte clusters in use (current/peak/max) >>>> 0/8/6144 mbuf 4096 byte clusters in use (current/peak/max) >>>> 0/8/6144 mbuf 8192 byte clusters in use (current/peak/max) >>>> 0/8/6144 mbuf 9216 byte clusters in use (current/peak/max) >>>> 0/8/6144 mbuf 12288 byte clusters in use (current/peak/max) >>>> 0/8/6144 mbuf 16384 byte clusters in use (current/peak/max) >>>> 0/8/6144 mbuf 65536 byte clusters in use (current/peak/max) >>>> 14176 Kbytes allocated to network (22% in use) >>>> 0 requests for memory denied >>>> 0 requests for memory delayed >>>> 0 calls to protocol drain routines >>>> >>>> >>>> something like kern.maxclusters=10000 or ? >>>> >>>> >>>> >>>> /Pete >>>> >>>> >>> >>> Only you can answer that, Pete. >>> >>> Try increasing it gradually until the errors go away. And if the >>> error returns, increase it again. If it makes your system unstable, >>> lower it until it returns to stability. Increments (and decrements, >>> if necessary) of 256 would probably be wise. >>> >>> Getting the right balance with any system is all about trial and error >>> - trying different things until things are running smoothly - or >>> acceptably so in some situations. It's also about the balance between >>> workability and stability. Sometimes you just can't have your cake >>> and eat it too - stability must be the priority. >>> >>> My $0.02 there. >>> >>> -- >>> Aaron Mason - Programmer, open source addict >>> I've taken my software vows - for beta or for worse >>> >> >> Hi, >> >> >> Indeed, the only problem is that if it's too low, then the system hangs. > > I guess only the network hangs. Since there is no clusters available to be > used by drivers or other sockets. Normaly the system should not hangup > itself because of that. > >> Presumably if it's too high, then the 'system instability' manifests itself >> has hanging too, so it's tricky to tell which way to go, once you deviate from >> the norm ... >> > > Yes, if set too high you can run out the kernel of memory (physical or > virtual) which is normaly causing a panic or freze. > >> Anyway for the archives I'm trying 8192 currently, hopefully that will reduce >> the crashes... >> >> >> 6016 mbufs in use: >> 2151 mbufs allocated to data >> 3860 mbufs allocated to packet headers >> 5 mbufs allocated to socket names and addresses >> 1979/5664/8192 mbuf 2048 byte clusters in use (current/peak/max) >> 0/8/8192 mbuf 4096 byte clusters in use (current/peak/max) >> 0/8/8192 mbuf 8192 byte clusters in use (current/peak/max) >> 0/8/8192 mbuf 9216 byte clusters in use (current/peak/max) >> 0/8/8192 mbuf 12288 byte clusters in use (current/peak/max) >> 0/8/8192 mbuf 16384 byte clusters in use (current/peak/max) >> 0/8/8192 mbuf 65536 byte clusters in use (current/peak/max) >> 14048 Kbytes allocated to network (38% in use) >> > > Your allocationg a max of 8192 2k buffers or 4096 4k pages or 16MB of > memory. On a modern system with > 1GB of memory everything below 64MB or > 128k clusters should work if you don't fiddle with other knobs that rob > all memory from the kernel. > > -- > :wq Claudio >
okay, sounds reasonable. I've also 'fiddled with other knobs' too, so I hope my kern.maxclusters at 8192 should not cause exhaustion conjunction with: net.inet.ip.ifq.maxlen=512 net.inet.tcp.recvspace=262144 net.inet.tcp.sendspace=262144 kern.maxfiles=8192 kern.maxclusters=8192 BTW, when the system runs out of (these?) resources, it sometimes prevents SSH access or squid use, but still keeps a CARP peering alive, preventing failover to it's backup partner, which is somewhat frustrating (I know I could script around this). On other occasions, it drops into ddb> , which at least allows the CARP backup to take over duties. (I know I should file a bug report for this) /Pete
