Re: split spamassasin servers
Run top on the machines running spamd. If load is high, but there are
regular amounts of CPU idling, then network tests are slowing the
processes down and your servers aren't working to their potential. In
which case, have more spamd children ready to handle more simultaneous
activity, which may require more ram. Load is just the number jobs in
the run queue, not the slowness of the server, some of the jobs could be
awaiting network traffic. Dual CPU machines handle higher load better
then single CPU machines. if CPU is always fully tied up with user
processes, then you need more CPU, or fewer tests.
At the mx level, reject mail that fails sbl-xbl tests, doesn't have
valid HELO/EHLO, and isn't for valid users. That will drastically reduce
the volume your SA servers have to process. Make them as picky as you
can without getting tarred and feathered by your end users.
On Wed, Feb 08, 2006 at 04:33:44PM +, Ronan wrote:
Im currently running 3 mailhubs into our uni which scan all mail.
I have two dual-opteron boxes running spamd 3.1 w/ DCC, razor, pyzor,
caching bind w/rbldnsd server for SURBL, {AWL,BAYES (running off
seperate MySQL DB)} and various rules from SARE.
The hubs scan the messags then route them to various hosts/domains.
the boxes are in failover atm and im loathed to simply round robin the
scanning to them as if one goes then were screwed, if no one is around
During busy periods of the day the mailhubs start refusing new
connections as the Spamd machine churns away on the existing emails and
cant keep up with the rate.
This is down purely to the network tests, becuase if I enable -L then
the mails simply flood in.
Im sure there are others out there who have had to draw the line between
what options they can include in their scanning to get the best stable
system vs performance.
What I had in mind is this:
At the MX level I simply run local tests only (we dont reject on
spamscore. we simply tag) and route the message as normal to our hosts.
Now on the hosts we could then run a version of SA without any of the
rules but simply a 'network only' version ie SURBL,razor,pyzor etc and
add whatever score is here to the headers in the message before
deliveing to the local users mailbox. As at this stage we are no longer
holding up any connections etc and the users can wait 10-20 extra
seconds for their message before the network tests finsih/timeout...
What modifications would be needed to SA to accomplish this or is this
an MTA issue to rewrite the headers on the hosts?
We run EXIM on all MTAs and hosts here so it shouldn't be too much of an
issue at that level.
What do you think?
Ronan
--
Ronan McGlue
Analyst / Programmer
CMC Systems Group
Queens University Belfast
--
/*
Jason Philbrook | Midcoast Internet Solutions - Internet Access,
KB1IOJ| Hosting, and TCP-IP Networks for Midcoast Maine
http://f64.nu/ | http://www.midcoast.com/
*/
chickenpox q
My .sig below sets off Chickenpox check #23. Can anyone help me find what it is in my .sig causes that so I can fix it? Thanks, Jason -- /* Jason Philbrook | Midcoast Internet Solutions - Internet Access, KB1IOJ| Hosting, and TCP-IP Networks for Midcoast Maine http://f64.nu/ | http://www.midcoast.com/ */
Re: Load Balancing with Postfix [and SpamAssassin]
. -- Bowie -- /* Jason Philbrook | Midcoast Internet Solutions - Internet Access, KB1IOJ| Hosting, and TCP-IP Networks for Midcoast Maine http://f64.nu/ | http://www.midcoast.com/ */
Re: OT: Mail/Spam Stats and MRTG
We uses these scripts with mrtg/postfix/clamav/spamassassin/procmail to sample the logfiles each time mrtg runs. mc1:/usr/local/mis/sbin # cat sacleanratio.mrtg #!/bin/bash tail -n 1000 /var/log/mail |grep spamd |grep clean message |wc -l |sed -e s/ tail -n 1000 /var/log/mail |grep spamd |grep seconds, |wc -l |sed -e s/ *//g echo 0 echo 0 mc1:/usr/local/mis/sbin # cat saspamratio.mrtg #!/bin/bash tail -n 1000 /var/log/mail |grep spamd |grep identified spam |wc -l |sed -e tail -n 1000 /var/log/mail |grep spamd |grep seconds, |wc -l |sed -e s/ *//g echo 0 echo 0 mc1:/usr/local/mis/sbin # cat satime.mrtg #!/bin/bash tail -n 5000 /var/log/mail |grep spamd |grep seconds |cut -d: -f5 |cut -d -ftdc -e 1000 `awk -f /usr/local/mis/sbin/avg.awk ~/num.txt` * p echo 0 echo 0 echo 0 mc1:/usr/local/mis/sbin # cat saratio.mrtg #!/bin/bash tail -n 1000 /var/log/mail |grep spamd |grep clean message |wc -l |sed -e s/ tail -n 1000 /var/log/mail |grep spamd |grep identified spam |wc -l |sed -e echo 0 echo 0 mc1:/usr/local/mis/sbin # more ../etc/mrtg/load.cfg WorkDir: /usr/local/apache/htdocs/mrtg WithPeak[_]: ymw #Options[_]: growright, gauge, nopercent, nolegend, nobanner, noo #AbsMax[_]: 40 XSize[_]: 500 YSize[_]: 160 Target[load]: `cat /proc/loadavg |cut -d -f1 ;echo 0 ; echo 0; echo 0` ShortLegend[load]: 1 min. YLegend[load]: CPU Load Options[load]: growright, gauge, nopercent, nolegend, nobanner, noo MaxBytes[load]: 30 Unscaled[load]: d Title[load]: CPU Load Analysis PageTop[load]: H3Load Analysis/H3 Target[spamd]: `/usr/local/mis/sbin/satime.mrtg` YLegend[spamd]: MilliSeconds Options[spamd]: growright, gauge, nopercent, nolegend, nobanner, noo ShortLegend[spamd]: Millisec MaxBytes[spamd]: 2 Title[spamd]: Spamd processing time averages PageTop[spamd]: H3spamd processing time average/H3 Target[cratio]: `/usr/local/mis/sbin/sacleanratio.mrtg` YLegend[cratio]: Messages Options[cratio]: growright, gauge, nopercent, nolegend, nobanner, dorelpercent, integer ShortLegend[cratio]: Messages Legend1[cratio]: Clean Messages Legend2[cratio]: Total Messages LegendI[cratio]: Clean Messages LegendO[cratio]: Total Messages MaxBytes[cratio]: 500 Title[cratio]: Clean versus total email PageTop[cratio]: H3Clean versus total email/H3 Target[sratio]: `/usr/local/mis/sbin/saspamratio.mrtg` YLegend[sratio]: Messages Options[sratio]: growright, gauge, nopercent, nolegend, nobanner, dorelpercent, integer ShortLegend[sratio]: Messages Legend1[sratio]: Spam Messages Legend2[sratio]: Total Messages LegendI[sratio]: Spam Messages LegendO[sratio]: Total Messages MaxBytes[sratio]: 500 Title[sratio]: Spam versus total email PageTop[sratio]: H3Spam versus total email/H3 On Mon, Jun 06, 2005 at 11:20:47AM -0400, Jake Colman wrote: Does anyone have any suggestions for using mrtg to produce a graph showing the amount of received email and how much of it was flagged as spam? I am using mrtg, sendmail, and procmail on all the same server. Thanks! ...Jake -- Jake Colman Sr. Applications Developer Principia Partners LLC Harborside Financial Center 1001 Plaza Two Jersey City, NJ 07311 (201) 209-2467 www.principiapartners.com -- /* Jason Philbrook | Midcoast Internet Solutions - Internet Access, KB1IOJ| Hosting, and TCP-IP Networks for Midcoast Maine http://f64.nu/ | http://www.midcoast.com/ */
Re: ALL_TRUSTED alteration
On the same topic... The SpamAssassin documentation doesn't describe this possibility, so this is why I ask the list for some clarification. I have a mix of private and public addresses on my network which can send email. I have the public addresses listed in trusted_networks like this: trusted_networks69.39.96.0/20 trusted_networks12.149.230.0/24 trusted_networks12.25.52.0/23 I'd like to add the private addresses we use too, but I'm not sure if that would open up to more spam. If I added 10.0.0.0/8 as a trusted network, I'm afraid it could let it spam sent from other organizations' private networks that relay through their normal public network mail servers or firewalls. Sort of like setting 192.168.0.0 might let in every infected computer's email behind simple home nat boxes. Which networks does trusted_networks apply to, as an internet path is really a whole bunch of networks? TIA, Jason On Thu, Jan 20, 2005 at 09:42:44AM -0500, Bowie Bailey wrote: From: Martin Hepworth [mailto:[EMAIL PROTECTED] Craig Zeigler wrote: I am getting very obvious spam through my SA filters. The only thing I think is that the value for ALL_TRUSTED is pushing it below the threshold. Where do I go to alter this test's effect on the spam count? I have searched through all of the .cf files in /usr/share/spamassassin and /etc/mail/spamassasin and can't figure it out. using SA version 3.0.1 add the following line to /etc/mail/spamassassin/local.cf score ALL_TRUSTED 0.0 This will turn off that rule completely. True, but a better idea is to configure SA so that the trust path works properly. Add some lines like the following to /etc/mail/spamassassin/local.cf to specify the networks and mailservers you control. trusted_networks 192.168.1.10 trusted_networks 172.16. You can add either networks, or single hosts. I prefer to add networks so that I don't have to reconfigure if I add or move a mailserver. These settings specify to SA which mailservers should be trusted. If you don't specify, it has to guess, and it doesn't work well with NATed networks. For more info: $ man Mail::SpamAssassin::Conf Bowie -- /* Jason Philbrook | Midcoast Internet Solutions - Internet Access, KB1IOJ| Hosting, and TCP-IP Networks for Midcoast Maine http://f64.nu/ | http://www.midcoast.com/ */
Re: any performance benefit to SA 3.0
Thanks for the input. I guess I should mention that we will likely be running this through mimedefang or some other sendmail milter instead of procmail and spamd. I understand that mimedefang has its own overhead and memory issues but I was wondering about the SA component. The reason I am more interested in a default configuration is we have many machines and individually customising SA for all the machines would take to much time, so we are likely to just pick one configuration and push it out to all our We use an rsync server to hold the master contents of the /etc/mail/spamassassin/ directory. All the servers doing mail processing have a crontab'd Nasty Bash Script to use rsync to update that directory from the rsync server, then restart spamd. We use the same network tests on every machine. I see no reason why anything in /etc/mail/spamassassin/ has to be specific to a particular hostname. machines. I was thinking of running without network tests because we have dnsbl enabled through sendmail before spamassassin runs, but if surbls don't add to much overhead we might turn on networks test but only for surbls. Steve Cohen -- /* Jason Philbrook | Midcoast Internet Solutions - Internet Access, KB1IOJ| Hosting, and TCP-IP Networks for Midcoast Maine http://f64.nu/ | http://www.midcoast.com/ */
