Re: Mbuf Clusters on 4.8
John Bdckstrand wrote: Ive been googling quite a bit now for problems with running out of mbuf clusters. Im basically sending a 30k datachunk down 1000-4000 connections, but 1000 is more than enough to quickly fill upp 8192 mbuf clusters. I also tried setting maximum amount of mbuf clusters to 65536, but that only made the box hard-wire 86MB of 96MB RAM, making it just as unsuable as a dead machine. It isn't the amount data you are sending but the overhead required for each network connection. I would recommend adding more RAM. Bump it up to 256MB. With mbuf set at 65536 and using 86MB, you should still have plenty left for your application. If that doesn't solve your problem or you can't add more memory, then you'll want to look at controling the number of simultaneous connections to a number that your box can handle. Of course, when the machine runs out of mbuf clusters, it dies. I also found this with google: Finally, the fact that FreeBSD 3.x panics when it runs out of mbuf clusters is a well-known problem. The solution is to not let it run out of mbuf clusters by configuring a sufficient number for them. From this it sounds as it is a problem that should be fixed, but it obviously isnt in 4.8. Is this behaviour now considered acceptable? And if so, doesnt this make FreeBSD extremely easy to kill using a simple DOS-attack? Is this fixed in any way on 5.1? Yup, that is what DoS attack is... exhaustion of one or more resources of the victim. P2P software is an easy way to exhaust mbuf buffers on a box. P2P software(e.g. edonkey) can be a useful network stress tool; opens lots of connections and pushes a lot of data. My experience with mbuf exhaustion on a 4-stable boxes has been the box basically loses network connectivty until it can recover some buffers. The box is still responsive from the console and killing the offending application from the console will free up the mbufs and restore network connectivity. good luck, greg ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Mbuf Clusters on 4.8
On Thu, Jun 26, 2003 at 03:03:20AM +0200, John B?ckstrand wrote: Ive been googling quite a bit now for problems with running out of mbuf clusters. Im basically sending a 30k datachunk down 1000-4000 connections, but 1000 is more than enough to quickly fill upp 8192 mbuf clusters. I also tried setting maximum amount of mbuf clusters to 65536, but that only made the box hard-wire 86MB of 96MB RAM, making it just as unsuable as a dead machine. Of course, when the machine runs out of mbuf clusters, it dies. I also found this with google: Finally, the fact that FreeBSD 3.x panics when it runs out of mbuf clusters is a well-known problem. The solution is to not let it run out of mbuf clusters by configuring a sufficient number for them. From this it sounds as it is a problem that should be fixed, but it obviously isnt in 4.8. Is this behaviour now considered acceptable? And if so, doesnt this make FreeBSD extremely easy to kill using a simple DOS-attack? Is this fixed in any way on 5.1? --- John B?ckstrand It's not panicking, it's running out of resources. Whenever you have this sort of problem you need to provide more information, there is absolutely no way I can help you like this. You need to, at the very minimum, give us 'netstat -m' output and make a serious attempt at figuring out what is consuming so many clusters. You could be running out of clusters but you could also be running out of memory before you run out of clusters, in which case you should probably _not_ increase nmbclusters and instead fix the underlying problem instead (re-work your application). In such a scenario, blindly bumping up nmbclusters can make the problem worse. Even if you had 'unlimited' (or dynamically growing) nmbclusters, you'd _still_ have the same problem and, what's more, it could actually render your system even more unusable as the machine would not be able to allocate memory for other more important uses. Yes, you are right, I didnt get a panick. Firstly, heres is /var/log/messages from when the box hung: Jun 26 02:29:30 sandbsd /kernel: All mbuf clusters exhausted, please see tuning(7). Jun 26 02:29:35 sandbsd last message repeated 4 times Jun 26 02:29:35 sandbsd /kernel: rl0: watchdog timeout Jun 26 02:29:36 sandbsd /kernel: All mbuf clusters exhausted, please see tuning(7). Jun 26 02:29:56 sandbsd last message repeated 17 times Jun 26 02:29:57 sandbsd /kernel: rl0: watchdog timeout I cant give netstat -m after it hung, obviously, but with 8192 max mbuf clusters I can see some time before the hang that peak is at 8192 clusters, I also saw lots of requests for memory denied, and requests for memory delayed, but no calls to drain routines. I also saw about 20MB allocated to network (60% of mb_map). This was the last time I tried hanging it, and I somehow thought it mas managing. After all, denying requests means it did actually cope with a memory starvation. The NIC is a realtek 8139. I wonder what significance those watchdog timeout messages have? I might try with a 3com later today (any card which doesnt require mbufs should be finer, if there is any such thing). --- John Bäckstrand ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Mbuf Clusters on 4.8
From this it sounds as it is a problem that should be fixed, but it obviously isnt in 4.8. Is this behaviour now considered acceptable? And if so, doesnt this make FreeBSD extremely easy to kill using a simple DOS-attack? Is this fixed in any way on 5.1? Yup, that is what DoS attack is... exhaustion of one or more resources of the victim. P2P software is an easy way to exhaust mbuf buffers on a box. P2P software(e.g. edonkey) can be a useful network stress tool; opens lots of connections and pushes a lot of data. My experience with mbuf exhaustion on a 4-stable boxes has been the box basically loses network connectivty until it can recover some buffers. The box is still responsive from the console and killing the offending application from the console will free up the mbufs and restore network connectivity. Ah, unfortunately my box doesnt respond even to keyboard events (caps lock etc). The behaviour you describe I find totally acceptable on the other hand. And the software Im writing happens to be p2p-related, but its not a edonkey server. :) --- John Bäckstrand ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
