Pete,
Wow, thank you for the explanation. I did let the persistent
server run for 30 min after I restarted the services. However, I did
stop the services, then started Sniffer service, then restart Imail
services. I could have gotten a backlog of retries at that moment that
pegged the CPU as you stated. We have batted around running BIND for
NT/2000 on the local machine, but my fear was overhead of another major
process running. I don't have any good stats on how much CPU/Memory
BIND on an Imail Server requires, thus, we have a SUN/BIND box local to
the switch. Are you aware of any stats on this?
We don't run the AVAFTERJM switch. This is done in part due to
so many of our customers still look at their spam email from time to
time. We heavily use the ROUTETO and MAILBOX command, thus, if I let a
virus go through to their to mailbox, they could potentially open a
virus spam email and hurt themselves.
We defrag each partition every night using Diskeeper and it
works great. I regularly look at the Sniffer directory to ensure no
left over .fin files and others that could cause server load. I will
retry it again tonight and see what type of results I get and post them
here. It could be as you say, I am on the far side :)
Thanks again,
Keith
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil
Sent: Friday, April 01, 2005 2:16 PM
To: Keith Johnson
Subject: Re[4]: [sniffer] Persistent Sniffer
On Friday, April 1, 2005, 11:44:07 AM, Keith wrote:
KJ> Pete,
KJ> Thanks for the reply.
KJ> Running on an IBM Xseries 225 Dual Xeon 2.4Ghz w/ 1GB RAM -
KJ> running IBM's ServerRAID 5i in IBM's RAID 10 config (4 73GB 10K
KJ> drives)
KJ> - O/S is Windows 2000 Standard Server SP4
KJ> Running Imail 8.15HF1 with Declude JM/Virus 1.82 - BIND DNS
KJ> Server is 1 hop away (on switch backbone). I had to drop back to
KJ> the non-persistent mode, thus the .stat file disappeared. I will
KJ> run it again tonight and copy the file away and post it here
tonight.
KJ> Thanks again for the time and aid.
I don't see any problems with this setup.
Your description sounds like your server is fairly heavily loaded
(35-55% cpu in peer-server mode), though I would expect more from the
hardware you've described.
I suspect that you may have run into the far side of the power curve
when you went to persistent server mode. In peer-server mode the failure
mode for overload conditions is much softer than with the persistent
peer server mode.
Up to the failure point in the power curve the persistent server mode
will provide a significant savings over peer-server, however once that
point is reached the persistent server mode tends to degrade much more
quickly and requires a significant drop in load before recovery occurs.
I'm working on some strategies to soften that curve a bit, but in the
mean time let's explore these options to get the best performance from
your server and reduce it's load. The we can see if the persistent
server engine will give you even more headroom:
1. I recommend running AVAFTERJM - are you doing this? Typically 80% or
more of email traffic is spam and so there is no good reason to attempt
a virus scan on these messages. If you hold messages and occasionally
re-insert them into the queue then they will not be scanned, however
there are ways to work around this when needed - and it is very likely
you would not re-insert a message that contained a virus anyway.
2. Consider running bind as a dns resolver on your mail server and
pointing the server to itself via the loopback address (127.0.0.1) for
DNS services. This tends to speed up processing significantly which also
reduces the number of message processes that are running at any given
time. YMMV, but I have seen this work consistently to improve
performance.
--- when trying persistent mode (minor adjustments really) ---
A. Set the Persistence value in your snflicid.cfg file to 3600. - no
need to check for a new rulebase every 10 minutes usually. These loop
events tear down the server momentarily which can perturb an otherwise
smooth running system when under heavy loads - thus minimizing the
frequency of these events may help.
B. Set LogFormat in your snflicid.cfg file to SingleLine. This provides
sufficient data for our purposes (most of the time) and should
significantly reduce the size of your log file.
C. Be sure to keep any unnecessary files out of the SNF working
directory - in particular you should clean out any orphaned files that
might still be lurking from previous crashes.
--- General ---
Be sure your drives are regularly defragmented.
Hope this helps,
_M
PS: I just had another random thought really --- Could it be that the
high CPU value was appropriate? If you had built up a queue of messages
to be processed then once the persistent server was put in place and the
system started processing messages again the CPU would probably be much
higher for a period of time until all of the backlog had been
eliminated. The persistent server would do its best to "nail up" at
least one of the CPUs until this was accomplished, so looking at the CPU
load during that period might not be the best way to understand the
situation. The CPU load would not drop back down again until the backlog
had be eliminated.
In comparison to the persistent server mode, the peer-server mode
imposes a greater restriction on message throughput and puts a higher
load on IO due to repeatedly loading the rulebase file. This can have
the effect of reducing the overall CPU load at the expense of raw
throughput under some circumstances. This, in fact, explains why the
peer-server mode has a softer performance failure curve than the
persistent server mode (in theory). Put another way, sometimes the
peer-server mode prevents the CPU from getting out of it's own way to
scan the messages - so the CPU load looks lower because it spends more
time waiting. In these cases putting the persistent server in place has
the effect of removing the obstacles so the CPU works harder and the
messages go through faster.
A better way to judge might be to check the overflow queue... the rate
at which it is being emptied (or slowness at which it grows) would
indicate a better throughput and that is probably the better goal.
-- just a random thought.
This E-Mail came from the Message Sniffer mailing list. For information
and (un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html
This E-Mail came from the Message Sniffer mailing list. For information and
(un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html
