On Tue, Jun 29, 2004 at 09:41:18AM -0700, wrote:
>>spamd can use a LOT of memory, if you let too many instances start
>>they will start consuming swap and appear to hang while shuttling ram
>>to disk, and getting more requests. I do it in smtp so I can limit
>>concurencyincoming but you may need to set --max-children
>
>The only thing I was able to glean from the logs is that most messages are
>taking about 30 seconds to scan. This doesn't seem right. Is that normal?
it's a factor of the message, your pre existing load, rule sets, cpu and
memory.
Check this out....
2004-06-29 19:39:03.390223500 info: setuid to qmaild succeeded
2004-06-29 19:39:04.946625500 checking message <[EMAIL PROTECTED]> for
qmaild:1004.
2004-06-29 19:39:54.384806500 identified spam (12.9/5.0) for qmaild:1004 in
52.4 seconds, 1574 bytes.
2004-06-29 19:40:11.872003500 connection from ohm [127.0.0.1] at port 41069
2004-06-29 19:40:11.893594500 info: setuid to qmaild succeeded
2004-06-29 19:40:12.006195500 checking message <[EMAIL PROTECTED]> for
qmaild:1004.
2004-06-29 19:40:19.523800500 identified spam (12.9/5.0) for qmaild:1004 in 7.7
seconds, 1555 bytes.
19:45:56 up 307 days, 3:48, 3 users, load average: 2.28, 1.67, 0.81
93 processes: 89 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 95.4% user, 4.6% system, 0.0% nice, 0.0% idle
Mem: 317884K total, 313424K used, 4460K free, 15020K buffers
Swap: 511992K total, 86984K used, 425008K free, 98128K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
4467 qmaild 20 0 80580 61M 8868 R 44.0 19.8 2:51 /usr/bin/perl -T
-w /usr/sbin/spamd -L --max-children 2 --syslog
=stderr
4477 qmaild 20 0 80576 61M 8904 R 44.0 19.8 2:32 /usr/bin/perl -T
-w /usr/sbin/spamd -L --max-children 2 --syslog
=stderr
4388 root 15 0 972 972 748 R 11.9 0.3 11:21 top id .8
2732 root 9 0 1012 768 680 R 0.0 0.2 3:32 /usr/sbin/sshd
I saw the mem free go from about 115 MB, down to 4.4 MB in no time...
for 11 minutes; and why? delivery of the first message in the log
above took ~50 seconds, then the remote timed out (or just decided to
re-deliver because the first delivery was rejected, this is in smtp). 17
seconds later, the same message id gets the same score in 7.7 seconds;
but residual spamd processes continue for ~11 minutes, consuming 100%
cpu and 111460 KB of ram in the process. The above top mem/cpu/process
profile continued until about a second or two before I took the below
top snapshot, when the spamd exited and instantly freed up the cpu and
111 Mb of ram.
19:51:29 up 307 days, 3:53, 3 users, load average: 1.81, 1.98, 1.22
91 processes: 90 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 6.6% user, 5.5% system, 0.0% nice, 87.9% idle
Mem: 317884K total, 201964K used, 115920K free, 14892K buffers
Swap: 511992K total, 86800K used, 425192K free, 95784K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
4388 root 19 0 972 972 748 R 11.0 0.3 11:59 top id .8
That 11 minutes of 100% cpu and almost swapping box, was for one message
that the remote apparently tried to deliver twice. The next entry in the
logs is for an unrelated message.
2004-06-29 20:02:34.459530500 connection from ohm [127.0.0.1] at port 41070
2004-06-29 20:02:34.481433500 info: setuid to qmaild succeeded
2004-06-29 20:02:34.507927500 checking message <[EMAIL PROTECTED]> for
qmaild:1004.
2004-06-29 20:02:38.175666500 clean message (1.1/5.0) for qmaild:1004 in 3.7
seconds, 768 bytes.
It's not clear why SA does this or what can be done to stop using all
those residual resources, after spamd logs and spamc exits. Not sure
if the same code is exercising here, but phenomena can be reliably
reproduced with "cat message | spamc -R -u smtpd | less"
I wonder if there is something about that message that is hard on SA?
It is attached as uuencoded file KMAMQDMXXVGHNIGCLYSH=hotmail.com.txt
Thanks for any assistance.
// George
FWIW - I don't do any bayes or rbl in SA.
--
George Georgalis, Architect and administrator, Linux services. IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
Key fingerprint = 5415 2738 61CF 6AE1 E9A7 9EF0 0186 503B 9831 1631
begin 600 KMAMQDMXXVGHNIGCLYSH=hotmail.com.txt
M1G)O;2!V=G1A87IJ971Z0&AO=&UA:6PN8V]M(%1U92!*=6X@,CD@,3DZ,SDZ
M,#`@,C`P-`I296-E:[EMAIL PROTECTED])O;2!C<&4P,#`S-#<S834X,F8M8VTP,&4P
M-F8Q8C!C9#8N8W!E+FYE="YC86)L92YR;V=E<G,N8V]M("@V.2XQ.34N.3<N
M,C(U*0H@(&)Y(&EU>'1A+F-O;2!W:71H(%--5%`[(#(Y($IU;B`R,#`T(#(S
M.C,Y.C`P("TP,#`P"E)E8V5I=F5D.B!F<F]M(#(Q,BXV.2XU,BXX."!B>2`V
M.2XQ.34N.3<N,C(U.R!4=64L(#(Y($IU;B`R,#`T(#(S.C(W.C`V("TP,3`P
M"DUE<W-A9V4M240Z(#Q+34%-441-6%A61TA.24=#3%E32$!H;W1M86EL+F-O
M;3X*1G)O;3H@(E(@2'5D9&QE<W1O;B(@/'9V=&%A>FIE='I`:&]T;6%I;"YC
M;VT^"E)E<&QY+51O.B`B4B!(=61D;&5S=&]N(B`\=G9T86%Z:F5T>D!H;W1M
M86EL+F-O;3X*5&\Z('1H;VUA<T!I=7AT82YC;VT*4W5B:[EMAIL PROTECTED]<Z(%)8
M(%9B>6%F8V=W<0I$871E.B!4=64L(#(Y($IU;B`R,#`T(#(R.C,S.C`V("TP
M,C`P"DU)344M5F5R<VEO;CH@,2XP"D-O;G1E;G0M5'EP93H@;75L=&EP87)T
M+V%L=&5R;F%T:79E.PH)8F]U;F1A<GD](BTM,#,X,3$Q,[EMAIL PROTECTED]
M(@H*+2TM+3`S.#$Q,3$Q.#0U-34Q.#4X.`I#;VYT96YT+51Y<&4Z('1E>'0O
M<&QA:6X["D-O;G1E;G0M5')A;G-F97(M16YC;V1I;F<Z('%U;W1E9"UP<FEN
M=&%B;&4*"D=E="!A;&P@>6]U($UE9'-I8V%T:6]N($YE961S('=I=&@@57,*
M"C$R($-A=&%L;V=U97,@-3`K(%!R;V1U8W1S(#$P,"[EMAIL PROTECTED];7,*"EEO=2!C
[EMAIL PROTECTED](&%L;"!Y;W5R(&UE9&EI8V%T:6]N<R!W:71H(&YO(&AA<W-S;&5S
M(&]R('!R;V)L96US+"!.;R!$;V-T=&]R/0IS(&%P<&]I;G1M96YT(&YE8V5S
M<V%R>[EMAIL PROTECTED];"!S:&5I<"!O=F5R;FEG:'[EMAIL PROTECTED]&\@>6]U"@[EMAIL
PROTECTED]&\@(&AT
M='`Z+R]W=W<N8V]V97)S=&]R>6YE=W,[EMAIL PROTECTED](&]U<B!S<&5C
M:6%L"@I3:&]O<"!W:71H(%5S(&9O<B!Y;W5R($UE961S($YE961S"@IR:6]R
M9&%N(&)A;&5E;B!B;VEL(&]N96ED82!N86EV971E('[EMAIL PROTECTED]')A9&EC
M=&]R>2!A;G1W97)P('[EMAIL PROTECTED])C83T*=&4@;6%R:V5T(&-H968@<'5I
M<W-A;[EMAIL PROTECTED];W(@9'5P;&EC:71Y(')A:60@:&]O<"!L96=E<F1E;6%I
M;B!P;V=R;[EMAIL PROTECTED]"G=A<[EMAIL
PROTECTED]<V%T:6]N(&%U8G)E>2!F;W)S>71H92!T
M<F%V97)S86P@<W!A<[EMAIL PROTECTED]:6YA('-M871T97)I;F<@=7!S:61E(&-A/0IT
M:&]L:6-I<[EMAIL PROTECTED])E=#TR,`H*86UP;&ET=61E(&MN96QT(')I;W)D86X@
M9&5C:6QE(&]F9FEC:6%L9&]M(&-A9&%V97)O=7,@9&ES8W5S<V%N="!P97)C
M97!T:78]"[EMAIL PROTECTED])A;F1I;&]Q=65N="!O:&UM971E<B!A;&)E<G1O(&=N87-H
M(&)E;GIE;F4@:&5N9')I8VMS(')E<W1R86EN="!H;W5D:6YI/0H@:&%N9&QE
M('-P:[EMAIL PROTECTED])I;7-T;VYE/3(P"[EMAIL
PROTECTED]&\@:'1T<#HO+W=W=RYC;W9E<G-T
M;W)Y;F5W<RYC;VTO87=A>2YH=&UL(&EF('EO=2!A;')E861Y(&[EMAIL PROTECTED]&AE
M;0H*979A;F=E;"!U<'-T86ER(&)E8VME<B!J;VEN="!D(&1I96QD<FEN('-E
M;&5C=&UA;B!E;&5C=')O96YC97!H86QO9W)A<&@@;'D]"FQE(&-A8V]P:&]N
M:7-T(&UA=&-H8F]O:R!S>6QV86X],C`*"@HM+2TM,#,X,3$Q,3$X-#4U-3$X
'[EMAIL PROTECTED]"F-H
`
end
size 1627
