From: <[EMAIL PROTECTED]>
> >
> >
> > On Mon, Jun 23, 2003 at 11:19:25AM +1000, James Gray wrote:
> >
> > > We are running into problems when we get a flood of
> > messages (>50/minute)
> > > as the whole mail filtering/scanning thing quickly chews up
> > all CPU time
> >
> > Are you running SpamAssassin as a daemon (spamd)? I had this problem
> > when I first setup SpamAssassin because I was using the perl program
> > (spamassassin) to process each message. I changed to using
> > spamc/spamd
> > and it now has negligible impact on the cpu load.
> >
> >
> > Cheers,
> >
> > John
>
> Here's the pertinent details for the system concerned:
> # ps aux | awk '{print $1 "\t" $3 "\t" $4 "\t" $10 "\t" $11}'
> USER %CPU %MEM TIME COMMAND
> nobody 78.1 14.7 0:11.69 /usr/bin/spamd
> root 0.0 0.0 0:28.50 (pagedaemon)
> root 0.0 0.0 0:05.64 (vmdaemon)
> root 0.0 0.0 0:11.99 (bufdaemon)
> root 0.0 0.0 0:11.79 (vnlru)
> root 0.0 0.0 8:24.99 (syncer)
> root 0.0 0.3 0:39.22 /usr/bin/perl
> bind 0.0 1.6 39:49.58/usr/local/sbin/named
> root 0.0 0.3 1:13.24 sendmail:
> root 0.0 9.4 0:03.92 /usr/bin/spamd
> root 0.0 0.5 0:00.01 sendmail:
> root 0.0 0.5 0:00.01 sendmail:
> root 0.0 0.5 0:00.01 sendmail:
> root 0.0 0.6 0:00.03 sendmail:
> root 0.0 0.5 0:00.01 /usr/bin/spamc
> root 0.0 0.6 0:00.01 sendmail:
> root 0.0 0.0 0:00.12 (swapper)
> root 0.0 0.0 0:25.62 /sbin/init
>
> (non-relevant processes snipped - sshd/csh/sh etc).
>
> Notice the spamd load? This looks a little high to me. But our spam
> rules are huge (the normal rules that come with Spamassassin + 1168
> custom rules). Those custom rules round out to 3504 lines of
> RULE/DESCRIPTION/SCORE..... so relatively large. FWIW we're running
> spamassassin 2.55.
>
> I can send anything that might shed more light on the problem
> (sendmail.cf exerpts etc)....just let me know :-)
>
Observations from your stats:
1. Over 6Gb of data, 10,000+ of emails, and 50+ emails/minute during
peak hours.
2. Two instances of /usr/bin/spamd with 78.1/14.7 CPU/MEM on
one and 0.0/9.4 CPU/MEM on the second. One instance of
/usr/bin/spamc is 0.0/0.5 CPU/MEM.
3. Five instances of sendmail with not one exceeding 0.0/0.6
CPU/MEM.
4. Swapper with almost not activity.
Comments
1. It appears that your system is processing at an average and
sustained rate of 30Mb per minute. This is quite good throughput.
2. I can see that /usr/bin/spamd is not multi-threading nicely as
one instance is extremely active and aggressive whilst the
second instance is not processing anything although it is
concurrently loaded in memory.
3. Sendmail MTA which you have identified to replace are
not actually chewing up too much resources as indicated by
five idle processes.
4. Physical memory is sufficient as indicated by the Swapper
process having done almost nothing.
5. In view of the above, I would suggest that Computing
Power is less than adequate for the amount of work that you
have at this moment and it is not so much your current
mail and accessories software that are directly the cause
for this inadequacy.
6. In view of the above, I would suggest further that there
are two ways to resolve your situation, namely:
a. Increase the CPU power of your box even whilst you
contemplate replacing or re-arranging your software.
This is the simplest and straight forward solution as
you know.
b. The other solution is, to use your Network as your
Computer. By this, I mean use two or several computers
connected by a networking technology such as TCP/IP
to provide computer processing to do a job. This is
especially useful for applications like Email Services
as it gives you flexibility and scalability as you go upwards
or downwards with your load. You can add or remove
computer/computers in your Network Computer
depending on your requirements. There is little that you
have to change in your network except for the
re-arrangement of your BIND configuration specifically
your MX records.
With your stats, it might also help if you have a look at
your vmstat, like
vmstat -n 30
I hope you will be able to alleviate your problems
quickly and in the process have some fun.
http://www.acay.com.au/~oscarp/disclaimer.html
--
SLUG - Sydney Linux User's Group - http://slug.org.au/
More Info: http://lists.slug.org.au/listinfo/slug
