On Mon, Jun 10, 2002 at 12:07:14AM +0100, Sean Rima wrote: | On Sun, 9 Jun 2002, Derrick Hudson muttered drunkenly: | | >| Yeah the spam[c/d] setup. My average is around 15 seconds, well it is | >| an old p133 the slowest appears to be 93 seconds. I am a dialup user | >| and when I go online off peak for the first time, fetchmail can throw | >| over a 1000 emails at spam[c/d]. I also use DCC and have no | >| performance problems there. | > | > The real problem you have is not so much spamd's performance, but the | > fact that it sits idle 99% of the time, then gets slammed with 1K | > messages in a matter of seconds. Even with my Duron 750 and not much | > else happening, the box can effectively freeze (no UI response) with a | > load average of 30 if I hit it with 900+ messages almost | > instantaneously. | | This is very true, 99% of the time spamd sits there waiting for | something to happen and then suddenly the poop hits the fan and | everything goes haywire. | | But if I disable spamc in Exim (I used your example config) and leave | dcc only I never get above a load average of 2.3 and dccd is flooding | with remote servers as well, pointing out that I use spamd with -L.
spamd is CPU bound. dcc is I/O (network) bound. You won't see a
system load from processes that are blocked on I/O.
| > What you should do is
| > 1) put this in your exim.conf (IIRC you're using exim)
| > deliver_queue_load_max = 5.0
| > 2) At least while you're retrieving your mail, run queue runners
| > quite often. I think the debian default is every 15 minutes.
| >
| > What this will do is cause exim to only queue the messages, no
| > delivery processing (SA scanning), if the system's load average is
| > above 5. When the load average drops below the threshold, the next
| > queue runner will attempt a delivery. By doing this you can spread
| > the load over a larger amount of time and not feel the effects as
| > much.
| >
| > Eg my system is "always on", so mail arrives as it arrives (usually
| > one at a time) so SA has no difficulty snagging a bit of CPU for a
| > couple seconds even while I'm doing all sorts of other work on the
| > machine.
|
| I have your setup in place and using the router with a domains option to
| restrict to to a couple of domains that I collect and forward to another
| remote box, one of these averages around 5 emails per collection and
| spamd will bring BSOD over the load average of 5.
Adjust the deliver_queue_load_max to suit.
| Strange thing is that I have a couple of those dyn dns entries and
| somethings I will get 3 or 4 spam mails at once, these also hit
| performance. but the real problem is that I need something better than a
| p133 to host spamd :)
Sure, for a CPU bound process the best thing you can do is give it
more CPU (or reduce the process' need for CPU).
See what happens if you add the load limiting options into your exim
config, but pick a reasonable load for your system. Choose a value
that will let spamd process stuff as soon as is reasonable, but that
also won't kill your system.
Another alternative, if you aren't collecting mail all that often, is
to just let it hit the roof and take a coffee break :-). That's what
I had to do that time I fed 900+ messages into exim (without that
option set) and could no longer get any UI response at all. After the
dust settled I saw a load average of 30, but who knows what it was
before that. There was no harm done that time. Now, I do have
another box that I've pulled all mail handling from (it didn't even
have SA on it) because with only 8MB RAM it would thrash too much and
the kernel would start killing processes. On that system no mail
handling is appropriate.
HTH,
-D
--
"He is no fool who gives up what he cannot keep to gain what he cannot lose."
--Jim Elliot
GnuPG key : http://dman.ddts.net/~dman/public_key.gpg
msg06086/pgp00000.pgp
Description: PGP signature
