Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Michael Amster wrote: Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA Just out of curiosity, are you seeing spamd suck up a lot of cpu? If so, it could be expiring bayes tokens. Networking delays (waits) are a good guess though. How about DNS? Which caching nameserver are you running? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
I had been experiencing a similar problem, SMTP sessions time out when trying to send mail through my server. I narrowed the problems down to the RBL's as well. I am currently running only the following: -r relays.ordb.org -r opm.blitzed.org -r cbl.abuseat.org. Adding additional blacklists will sometimes dramatically slow down the beginning of the SMTP transaction. Telnet to your server on port 25 and watch how long it takes for the HELO message. Mine went from 15+ seconds down to 1. It also may be worth checking that your DNS configuration is correct. -Jason -Original Message- From: Michael Amster [mailto:[EMAIL PROTECTED] Sent: Monday, October 30, 2006 2:31 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Slow/hung smtpd with Qmailtoaster Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Eric Shubes wrote: Michael Amster wrote: Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA Just out of curiosity, are you seeing spamd suck up a lot of cpu? If so, it could be expiring bayes tokens. No, this machine is nearly idling. I can check on the bayes - I think I must have a huge bayes database now. Is there a way to check it or to manually expire them and then resume them? Networking delays (waits) are a good guess though. How about DNS? Which caching nameserver are you running? I am running DJB DNS - I have not seen delays on DNS, but have not done many queries. - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
DNS is correct - running a local djbdns and it is screaming. I have heard such good things about sbl-xbl.spamhaus.org that I wanted to keep it. Did you strike it from your list or just not consider it? jason p wrote: I had been experiencing a similar problem, SMTP sessions time out when trying to send mail through my server. I narrowed the problems down to the RBL's as well. I am currently running only the following: -r relays.ordb.org -r opm.blitzed.org -r cbl.abuseat.org. Adding additional blacklists will sometimes dramatically slow down the beginning of the SMTP transaction. Telnet to your server on port 25 and watch how long it takes for the HELO message. Mine went from 15+ seconds down to 1. It also may be worth checking that your DNS configuration is correct. -Jason -Original Message- From: Michael Amster [mailto:[EMAIL PROTECTED] Sent: Monday, October 30, 2006 2:31 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Slow/hung smtpd with Qmailtoaster Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Michael Amster wrote: Eric Shubes wrote: Michael Amster wrote: Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA Just out of curiosity, are you seeing spamd suck up a lot of cpu? If so, it could be expiring bayes tokens. No, this machine is nearly idling. I can check on the bayes - I think I must have a huge bayes database now. Is there a way to check it or to manually expire them and then resume them? Check: # spamassassin -D bayes --lint Manually expire: # sa-learn --force-expire Networking delays (waits) are a good guess though. How about DNS? Which caching nameserver are you running? I am running DJB DNS - I have not seen delays on DNS, but have not done many queries. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Here's what I'm running, FWIW: -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org -r sbl-xbl.spamhaus.org Not having a problem on a high volume server with this. Michael Amster wrote: DNS is correct - running a local djbdns and it is screaming. I have heard such good things about sbl-xbl.spamhaus.org that I wanted to keep it. Did you strike it from your list or just not consider it? jason p wrote: I had been experiencing a similar problem, SMTP sessions time out when trying to send mail through my server. I narrowed the problems down to the RBL's as well. I am currently running only the following: -r relays.ordb.org -r opm.blitzed.org -r cbl.abuseat.org. Adding additional blacklists will sometimes dramatically slow down the beginning of the SMTP transaction. Telnet to your server on port 25 and watch how long it takes for the HELO message. Mine went from 15+ seconds down to 1. It also may be worth checking that your DNS configuration is correct. -Jason -Original Message- From: Michael Amster [mailto:[EMAIL PROTECTED] Sent: Monday, October 30, 2006 2:31 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Slow/hung smtpd with Qmailtoaster Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Eric Shubes wrote: Michael Amster wrote: Eric Shubes wrote: Michael Amster wrote: Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA Just out of curiosity, are you seeing spamd suck up a lot of cpu? If so, it could be expiring bayes tokens. No, this machine is nearly idling. I can check on the bayes - I think I must have a huge bayes database now. Is there a way to check it or to manually expire them and then resume them? Check: # spamassassin -D bayes --lint Manually expire: # sa-learn --force-expire Looks good to me: [EMAIL PROTECTED] control]# spamassassin -D bayes --lint [11083] dbg: bayes: tie-ing to DB file R/O /home/vpopmail/.spamassassin/bayes_toks [11083] dbg: bayes: tie-ing to DB file R/O /home/vpopmail/.spamassassin/bayes_seen [11083] dbg: bayes: found bayes db version 3 [11083] dbg: bayes: DB journal sync: last sync: 1162238883 [11083] dbg: bayes: DB journal sync: last sync: 1162238883 [11083] dbg: bayes: corpus size: nspam = 4672712, nham = 162872 [11083] dbg: bayes: header tokens for *F = U*ignore D*compiling.spamassassin.taint.org D*spamassassin.taint.org D*taint.org D*org [11083] dbg: bayes: header tokens for *m = 1162238930 lint_rules [11083] dbg: bayes: header tokens for X-Spam-Relays-External = [11083] dbg: bayes: header tokens for X-Spam-Relays-Internal = [11083] dbg: bayes: header tokens for *RT = [11083] dbg: bayes: header tokens for *RU = [11083] dbg: bayes: token 'H*Ad:D*org' = 0.152146823350312 [11083] dbg: bayes: score = 0.0929143971693591 [11083] dbg: bayes: DB journal sync: last sync: 1162238883 [11083] dbg: bayes: untie-ing [11083] dbg: bayes: untie-ing db_toks [11083] dbg: bayes: untie-ing db_seen [11083] info: rules: meta test SARE_HEAD_SUBJ_RAND has undefined dependency 'SARE_XMAIL_SUSP2' [11083] info: rules: meta test SARE_HEAD_SUBJ_RAND has undefined dependency 'SARE_HEAD_XAUTH_WARN' [11083] info: rules: meta test SARE_RD_SAFE has undefined dependency 'SARE_RD_SAFE_MKSHRT' [11083] info: rules: meta test SARE_RD_SAFE has undefined dependency 'SARE_RD_SAFE_GT' [11083] info: rules: meta test SARE_RD_SAFE has undefined dependency 'SARE_RD_SAFE_TINY' [11083] info: rules: meta test FP_MIXED_PORN3 has undefined dependency 'FP_PENETRATION' [EMAIL PROTECTED] control]# sa-learn --force-expire bayes: synced databases from journal in 0 seconds: 2074 unique entries (3793 total entries) the sa-learn is taking a while... - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Eric Shubes wrote: Michael Amster wrote: Eric Shubes wrote: Michael Amster wrote: Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA Just out of curiosity, are you seeing spamd suck up a lot of cpu? If so, it could be expiring bayes tokens. No, this machine is nearly idling. I can check on the bayes - I think I must have a huge bayes database now. Is there a way to check it or to manually expire them and then resume them? Check: # spamassassin -D bayes --lint Manually expire: # sa-learn --force-expire Here is the complete output: [EMAIL PROTECTED] control]# sa-learn --force-expire bayes: synced databases from journal in 0 seconds: 2074 unique entries (3793 total entries) expired old bayes database entries in 109 seconds 430987 entries kept, 163306 deleted token frequency: 1-occurrence tokens: 55.33% token frequency: less than 8 occurrences: 29.25% - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
I removed sbl-xbl.spamhaus.org a few weeks ago when their legal troubles started; it was causing me SMTP timeout problems back then. I tried putting it back once about a week ago, and as soon as I did my SMTP connections started taking forever. I also removed the others because they either didn't respond to pings (not a big deal, many servers don't) or they didn't resolve to an IP address (I don't think that is good). I would try sticking just the servers that you know are good in there first, then add the others one at a time. -Jason -Original Message- From: Michael Amster [mailto:[EMAIL PROTECTED] Sent: Monday, October 30, 2006 2:58 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster DNS is correct - running a local djbdns and it is screaming. I have heard such good things about sbl-xbl.spamhaus.org that I wanted to keep it. Did you strike it from your list or just not consider it? jason p wrote: I had been experiencing a similar problem, SMTP sessions time out when trying to send mail through my server. I narrowed the problems down to the RBL's as well. I am currently running only the following: -r relays.ordb.org -r opm.blitzed.org -r cbl.abuseat.org. Adding additional blacklists will sometimes dramatically slow down the beginning of the SMTP transaction. Telnet to your server on port 25 and watch how long it takes for the HELO message. Mine went from 15+ seconds down to 1. It also may be worth checking that your DNS configuration is correct. -Jason -Original Message- From: Michael Amster [mailto:[EMAIL PROTECTED] Sent: Monday, October 30, 2006 2:31 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Slow/hung smtpd with Qmailtoaster Hi: I run a relatively lightly loaded qmailtoaster - pretty current. I have noticed that after a while the qmail-smtpd process seems to get bogged down accepting incoming messages. I run rbl and my blacklists file is: -r sbl-xbl.spamhaus.org -r bl.spamcop.net -r relays.ordb.org -r dnsbl.antispam.or.id -r opm.blitzed.org -r list.dsbl.org -r cbl.abuseat.org I am wondering if any of these might be timing out or causing the issues - any recommendations on the best list to use to leverage good rbl filtering without causing resource hangups? If there is another place to look at the slowdowns, I can check too - my gut instinct is that it is in the network checks (I also have Pyzor and Razor2 enabled) -MA - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
jason p wrote: I removed sbl-xbl.spamhaus.org a few weeks ago when their legal troubles started; it was causing me SMTP timeout problems back then. I tried putting it back once about a week ago, and as soon as I did my SMTP connections started taking forever. I also removed the others because they either didn't respond to pings (not a big deal, many servers don't) or they didn't resolve to an IP address (I don't think that is good). I would try sticking just the servers that you know are good in there first, then add the others one at a time. Very few BL's will be ping-able, or resolve. They're essentially DNS servers. Your machine makes a query to them (using this email as an example) asking who qmailtoaster.com is. If the BL server returns 127.0.0.1 (this result can change from BL to BL) then your machine knows it's good. If it returns 127.0.0.2 then it knows it's bad and can bounce it. - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Jake Vickers wrote: jason p wrote: I removed sbl-xbl.spamhaus.org a few weeks ago when their legal troubles started; it was causing me SMTP timeout problems back then. I tried putting it back once about a week ago, and as soon as I did my SMTP connections started taking forever. I also removed the others because they either didn't respond to pings (not a big deal, many servers don't) or they didn't resolve to an IP address (I don't think that is good). I would try sticking just the servers that you know are good in there first, then add the others one at a time. Very few BL's will be ping-able, or resolve. They're essentially DNS servers. Your machine makes a query to them (using this email as an example) asking who qmailtoaster.com is. If the BL server returns 127.0.0.1 (this result can change from BL to BL) then your machine knows it's good. If it returns 127.0.0.2 then it knows it's bad and can bounce it. EE wrote a nice explanation on the list too a little while back. Check the archives. -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
I performed an experiment to see what happens - I removed all the RBLs and saw that the response decayed from instant response after a qmailctl restart to about 15s after about 1hr. of running. It looks like the RBLs have no impact on my response time. This points to bayes expiration as an issue. my relevant spamassassin local.cf: ok_locales all skip_rbl_checks 0 report_safe 1 lock_method flock rewrite_header Subject [SPAM] required_score 6.0 bayes_file_mode 0666 auto_whitelist_path /home/vpopmail/.spamassassin/auto-whitelist use_pyzor 1 use_razor2 1 use_dcc 1 use_auto_whitelist 1 use_bayes 1 bayes_auto_learn 1 use_bayes_rules 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 bayes_path /home/vpopmail/.spamassassin/bayes -MA Eric Shubes wrote: Jake Vickers wrote: jason p wrote: I removed sbl-xbl.spamhaus.org a few weeks ago when their legal troubles started; it was causing me SMTP timeout problems back then. I tried putting it back once about a week ago, and as soon as I did my SMTP connections started taking forever. I also removed the others because they either didn't respond to pings (not a big deal, many servers don't) or they didn't resolve to an IP address (I don't think that is good). I would try sticking just the servers that you know are good in there first, then add the others one at a time. Very few BL's will be ping-able, or resolve. They're essentially DNS servers. Your machine makes a query to them (using this email as an example) asking who qmailtoaster.com is. If the BL server returns 127.0.0.1 (this result can change from BL to BL) then your machine knows it's good. If it returns 127.0.0.2 then it knows it's bad and can bounce it. EE wrote a nice explanation on the list too a little while back. Check the archives. - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [qmailtoaster] Slow/hung smtpd with Qmailtoaster
Michael Amster wrote: I performed an experiment to see what happens - I removed all the RBLs and saw that the response decayed from instant response after a qmailctl restart to about 15s after about 1hr. of running. It looks like the RBLs have no impact on my response time. This points to bayes expiration as an issue. I wouldn't draw that conclusion, but it's certainly something to look into. my relevant spamassassin local.cf: ok_locales all skip_rbl_checks 0 report_safe 1 lock_method flock rewrite_header Subject [SPAM] required_score 6.0 bayes_file_mode 0666 auto_whitelist_path /home/vpopmail/.spamassassin/auto-whitelist use_pyzor 1 use_razor2 1 use_dcc 1 use_auto_whitelist 1 use_bayes 1 bayes_auto_learn 1 use_bayes_rules 1 bayes_auto_learn_threshold_spam 7.0 bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_expire 1 bayes_path /home/vpopmail/.spamassassin/bayes -MA You can set bayes_auto_expire 0 (and restart SA) to turn auto-expiration off. I'm not convinced that'll solve your problem though. Your manual expire run appeared to take 109 seconds. Since running that, you shouldn't be suffering from auto-expire again for at least 12 hours (see EXPIRATION section of man sa-learn). I'd still be inclined to turn it off though and do it with a daily cron job. You might also be experiencing RBL delays via SA if you have SURBLs enabled. To check that, you might turn local-only rules back on (-L option, see wiki SURBL). Are you seeing a heavy cpu use by SA? How long do scans typically take? -- -Eric 'shubes' - QmailToaster hosted by: VR Hosted http://www.vr.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
