On 05/20/2011 11:54 AM, Eric Shubert wrote:
On 05/19/2011 12:22 PM, Jake Vickers wrote:
This is more of an interest than anything - I have a server where I
actually have enough domains and users on it that the ATA/133 spindle
can't keep up with the mail flow.
Has anyone else encountered a situation where they have hit a physical
spindle limitation? Has anyone ever seen a page or collected data to
show how many IO/s a spindle can handle? I have some numbers, but
they're for FCAL, which is not the case for this particular server.
Just curious if anyone else has pushed they system to the edge of
hardware limitations before.
---------------------------------------------------------------------------------
I have not. However, I have maxed out the system bus, which happened
well before the HDD reached full capacity. South bridge to be
specific. I would expect that to be where the bottleneck is. Given
that the SB handles all PCI devices, I would expect that optimizing
network traffic could open up a little bandwidth for disk i/o. Here is
a recent post from Richard Wilson on the Phoenix LUG list that I'm
interested to try:
All,
I ran into this some time ago and thought I should share it -- on a
server running iptables that's got a heavy network load, these can
make a big difference.
Netfilter (iptables is the main interface command) is a stateful
firewall so it tracks connections. This means that it allocates a
certain amount of real RAM when the system boots to hold the
Connection Tracking Table. Resizing this table requires a reboot.
Something you NEVER want to see is console messages that say:
ip_conntrack: table full, dropping packet
When I ran into these, I had to do some digging, and found out how to
resize the conntrack table, as well as how to decrease the time out
value for connections (default for Red Hat is 5 days!). Changing the
time out value doesn't require a reboot; changing the table size to
an appropriate value for your workload does.
Under Red Hat/Fedora/Centos, in /etc/sysctl.conf (takes effect at
boot time):
# Set Conntrack time out to 12 hours
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 43200
# Set the size of the conntrack table (old and new versions):
net.ipv4.ip_conntrack_max = XXXXXXX
net.ipv4.netfilter.ip_conntrack_max = XXXXXXX
Dynamically change the time out value on the running system (note
spaces around the = in the sysctl file, but not here!):
sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=43200
These helped, but what really made the biggest difference was this:
I found that the conntrack table had a LOT of entries for traffic
from 127.0.0.1 (!!) Yes, netfilter will track ALL network traffic.
If your kernel is 2.6.9 or above, this command should work to tell
netfilter to not track localhost traffic:
iptables -t raw -A OUTPUT -o lo -j NOTRACK
After doing this on several mail relay servers we saw the number of
entries in the conntrack table drop to about half what they had
been. Of course, they don't drop right away -- existing table
entries have to time out.
Hopefully this may save some of you some headaches.
Richard Wilson
Thanks Eric. Not the same problem I'm running into, but good
information. I was not aware of the RHEL 5 day timeout. Interesting.
My scenario is by using a RAMdisk for email scanning, I've reduced the
scan time by 1000%. All good, except now the system accepts and scans
too many messages and can't flush them to the spindle fast enough. I end
up with thousands of messages waiting to be delivered to local account
hung in the queue. I downsized the size of the RAMdisk to throttle this
(when it fills up it can't accept new messages and tells the remote to
try later), but this is not ideal. I'm in remedial now to move to a fast
server - was on a P4 3.2G with 2G of RAM and an ATA/133 disk. Moving to
something a little more robust, with faster disks :)
Anyway, was curious as to if anyone else had hit similar situations.
What was the scenarios where you over ran the bus?
---------------------------------------------------------------------------------
Qmailtoaster is sponsored by Vickers Consulting Group
(www.vickersconsulting.com)
Vickers Consulting Group offers Qmailtoaster support and installations.
If you need professional help with your setup, contact them today!
---------------------------------------------------------------------------------
Please visit qmailtoaster.com for the latest news, updates, and packages.
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
