On Mon, Mar 01, 2010 at 12:15:39PM +0100, Pete Vickers wrote:
> On 26. feb. 2010, at 11.58, Claudio Jeker wrote:
>
> > On Fri, Feb 26, 2010 at 11:30:30AM +0100, Pete Vickers wrote:
> >> On 26. feb. 2010, at 03.01, Aaron Mason wrote:
> >>
> >>> On Thu, Feb 25, 2010 at 10:04 AM, Pete Vickers <[email protected]> wrote:
> >>>> Hi,
> >>>>
> >>>> A proxy (squid) server running i368/4.6RELEASE with around 800 users,
> what
> >>>> would be a reasonable value to increase kern.maxclusters too, to cure
> this
> >> :
> >>>>
> >>>>
> >>>> r...@proxy-s ~> grep mcl /var/log/messages
> >>>> Dec 10 10:13:43 proxy-s /bsd: WARNING: mclpools limit reached; increase
> >>>> kern.maxclusters
> >>>> Dec 10 11:06:07 proxy-s /bsd: WARNING: mclpools limit reached; increase
> >>>> kern.maxclusters
> >>>> Dec 15 13:41:48 proxy-s /bsd: WARNING: mclpools limit reached; increase
> >>>> kern.maxclusters
> >>>>
> >>>>
> >>>> r...@proxy-s ~> sysctl kern.maxclusters
> >>>> kern.maxclusters=6144
> >>>>
> >>>>
> >>>> r...@proxy-s ~> netstat -m
> >>>> 4098 mbufs in use:
> >>>> 1131 mbufs allocated to data
> >>>> 2962 mbufs allocated to packet headers
> >>>> 5 mbufs allocated to socket names and addresses
> >>>> 1084/6152/6144 mbuf 2048 byte clusters in use (current/peak/max)
> >>>> 0/8/6144 mbuf 4096 byte clusters in use (current/peak/max)
> >>>> 0/8/6144 mbuf 8192 byte clusters in use (current/peak/max)
> >>>> 0/8/6144 mbuf 9216 byte clusters in use (current/peak/max)
> >>>> 0/8/6144 mbuf 12288 byte clusters in use (current/peak/max)
> >>>> 0/8/6144 mbuf 16384 byte clusters in use (current/peak/max)
> >>>> 0/8/6144 mbuf 65536 byte clusters in use (current/peak/max)
> >>>> 14176 Kbytes allocated to network (22% in use)
> >>>> 0 requests for memory denied
> >>>> 0 requests for memory delayed
> >>>> 0 calls to protocol drain routines
> >>>>
> >>>>
> >>>> something like kern.maxclusters=10000 or ?
> >>>>
> >>>>
> >>>>
> >>>> /Pete
> >>>>
> >>>>
> >>>
> >>> Only you can answer that, Pete.
> >>>
> >>> Try increasing it gradually until the errors go away. And if the
> >>> error returns, increase it again. If it makes your system unstable,
> >>> lower it until it returns to stability. Increments (and decrements,
> >>> if necessary) of 256 would probably be wise.
> >>>
> >>> Getting the right balance with any system is all about trial and error
> >>> - trying different things until things are running smoothly - or
> >>> acceptably so in some situations. It's also about the balance between
> >>> workability and stability. Sometimes you just can't have your cake
> >>> and eat it too - stability must be the priority.
> >>>
> >>> My $0.02 there.
> >>>
> >>> --
> >>> Aaron Mason - Programmer, open source addict
> >>> I've taken my software vows - for beta or for worse
> >>>
> >>
> >> Hi,
> >>
> >>
> >> Indeed, the only problem is that if it's too low, then the system hangs.
> >
> > I guess only the network hangs. Since there is no clusters available to be
> > used by drivers or other sockets. Normaly the system should not hangup
> > itself because of that.
> >
> >> Presumably if it's too high, then the 'system instability' manifests
> itself
> >> has hanging too, so it's tricky to tell which way to go, once you deviate
> from
> >> the norm ...
> >>
> >
> > Yes, if set too high you can run out the kernel of memory (physical or
> > virtual) which is normaly causing a panic or freze.
> >
> >> Anyway for the archives I'm trying 8192 currently, hopefully that will
> reduce
> >> the crashes...
> >>
> >>
> >> 6016 mbufs in use:
> >> 2151 mbufs allocated to data
> >> 3860 mbufs allocated to packet headers
> >> 5 mbufs allocated to socket names and addresses
> >> 1979/5664/8192 mbuf 2048 byte clusters in use (current/peak/max)
> >> 0/8/8192 mbuf 4096 byte clusters in use (current/peak/max)
> >> 0/8/8192 mbuf 8192 byte clusters in use (current/peak/max)
> >> 0/8/8192 mbuf 9216 byte clusters in use (current/peak/max)
> >> 0/8/8192 mbuf 12288 byte clusters in use (current/peak/max)
> >> 0/8/8192 mbuf 16384 byte clusters in use (current/peak/max)
> >> 0/8/8192 mbuf 65536 byte clusters in use (current/peak/max)
> >> 14048 Kbytes allocated to network (38% in use)
> >>
> >
> > Your allocationg a max of 8192 2k buffers or 4096 4k pages or 16MB of
> > memory. On a modern system with > 1GB of memory everything below 64MB or
> > 128k clusters should work if you don't fiddle with other knobs that rob
> > all memory from the kernel.
> >
> > --
> > :wq Claudio
> >
>
>
> okay, sounds reasonable. I've also 'fiddled with other knobs' too, so I hope
> my kern.maxclusters at 8192 should not cause exhaustion conjunction with:
>
>
> net.inet.ip.ifq.maxlen=512
> net.inet.tcp.recvspace=262144
> net.inet.tcp.sendspace=262144
> kern.maxfiles=8192
> kern.maxclusters=8192
>
>
> BTW, when the system runs out of (these?) resources, it sometimes prevents SSH
> access or squid use, but still keeps a CARP peering alive, preventing failover
> to it's backup partner, which is somewhat frustrating (I know I could script
> around this). On other occasions, it drops into ddb> , which at least allows
> the CARP backup to take over duties. (I know I should file a bug report for
> this)
>
>
>
> /Pete
How many tcp connections are you serving? Your high values of
net.inet.tcp.recvspace and net.inet.tcp.sendspace will eat a lot of
memory resources: about half an M per connection. It could very welll
explain your failing ssh and squid connections.
-0tto
