Quoting [EMAIL PROTECTED]:
> >
> >
> > On Mon, Jun 23, 2003 at 11:19:25AM +1000, James Gray wrote:
> >
> > > We are running into problems when we get a flood of
> > messages (>50/minute)
> > > as the whole mail filtering/scanning thing quickly chews up
> > all CPU time
> >
> > Are you running SpamAssassin as a daemon (spamd)? I had this problem
> > when I first setup SpamAssassin because I was using the perl program
> > (spamassassin) to process each message. I changed to using
> > spamc/spamd
> > and it now has negligible impact on the cpu load.
> >
> >
> > Cheers,
> >
> > John
>
> Here's the pertinent details for the system concerned:
> # ps aux | awk '{print $1 "\t" $3 "\t" $4 "\t" $10 "\t" $11}'
> USER %CPU %MEM TIME COMMAND
> nobody 78.1 14.7 0:11.69 /usr/bin/spamd
> root 0.0 0.0 0:28.50 (pagedaemon)
> root 0.0 0.0 0:05.64 (vmdaemon)
> root 0.0 0.0 0:11.99 (bufdaemon)
> root 0.0 0.0 0:11.79 (vnlru)
> root 0.0 0.0 8:24.99 (syncer)
> root 0.0 0.3 0:39.22 /usr/bin/perl
> bind 0.0 1.6 39:49.58/usr/local/sbin/named
> root 0.0 0.3 1:13.24 sendmail:
> root 0.0 9.4 0:03.92 /usr/bin/spamd
> root 0.0 0.5 0:00.01 sendmail:
> root 0.0 0.5 0:00.01 sendmail:
> root 0.0 0.5 0:00.01 sendmail:
> root 0.0 0.6 0:00.03 sendmail:
> root 0.0 0.5 0:00.01 /usr/bin/spamc
> root 0.0 0.6 0:00.01 sendmail:
> root 0.0 0.0 0:00.12 (swapper)
> root 0.0 0.0 0:25.62 /sbin/init
>
> (non-relevant processes snipped - sshd/csh/sh etc).
>
> Notice the spamd load? This looks a little high to me. But our spam
> rules are huge (the normal rules that come with Spamassassin + 1168
> custom rules). Those custom rules round out to 3504 lines of
> RULE/DESCRIPTION/SCORE..... so relatively large. FWIW we're running
> spamassassin 2.55.
>
> I can send anything that might shed more light on the problem
> (sendmail.cf exerpts etc)....just let me know :-)
>
Your vmstat will provide additional info, like a
vmstat -n 30. It will be interesting to see what it shows.
Observing your stats:
1. Over 6 Gb of data processed each day.
1. Over 50 emails per minute translates to over 3000/hour
2. Only two processes for /usr/bin/spamd
with very unbalanced load of 78/0% CPU and 15/9% MEM.
3. Only one process for /usr/bin/spamc of 0.0/9.5% CPU/MEM.
Comments:
1. 6 Gb of data a day at 50 emails per minute is roughly
300K per email, assuming 10 hours of work. So, at peak
time your system is processing 15Mb/minute of data and is flat
out. The 15Mb/minute is quite acceptable.
2. Although two /usr/bin/spamd processes are spawned the
processing load is not distributed which appears to me
/usr/bin/spamd is not multi-threading although the second
process is concurrently loaded in memory as shown by 0% CPU
and 9.4% MEM utilisation.
3. In view that 15Mb/minute processing outcome is quite
acceptable even with a /usr/bin/spamd that is not multi-
threading, I would suggest it is time to add more computing
power to your project even whilst you contemplate re-arranging
your software.
4. There are two ways to add more computing power:
a. Increase the CPU power in a single box which is straight
forward but is limited in its scalability, or
b. Re-arrange the way your applications will compute by
utilising the Network as your Computer. By this I mean
utilising more than one computer connected on a network
to cooperate together to do the job. This is especially
an excellent strategy with Email processing and is
normally done as we scale up our load. This is excellent
solution I believe because it is extremely scalable for
email types of applications. We can quickly scale up or
down by adding or removing a computer/computers in the
work group.
I hope you can resolve your problems soon and hope that
in the process you have fun.
http://www.acay.com.au/~oscarp/disclaimer.html
-------------------------------------------------
This mail sent through IMP: http://horde.org/imp/
--
SLUG - Sydney Linux User's Group - http://slug.org.au/
More Info: http://lists.slug.org.au/listinfo/slug
