Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL?
I like having specific RBLs logged. I just installed spamdyke on a few qmail-toasters yesterday (replacing rblsmtpd), and was going to as about this. Michael beat me to it! ;) If simultaneous queries are being done, can all RBLs that match be logged? Perhaps a comma separated list within parenthesis. This would make it possible to gather stats on the effectiveness of the RBLs being used. Sam Clippinger wrote: Yes, this is certainly possible. Right now spamdyke identifies the RBL in its message to the remote server but not in the logs. Good idea! What would be a good way to log this information (preferably without breaking existing scripts)? I'm thinking as I type here, but spamdyke already follows the rejection reason with parenthesis (when the log level is high enough) to indicate which file/line matched for file-based filters... perhaps the same could be done for RBLs/RHSBLs. Something like this: DENIED_RBL_MATCH(rbl.example.com) As for reordering the RBLs to put the often-matched ones first, the next version of spamdyke will make that less necessary. By default, it will query all RBLs simultaneously, regardless of their order. (That behavior can be prevented with a new flag -- ordering would be important in that case.) -- Sam Clippinger Michael Colvin wrote: To find real numbers, you would have to consider how many connections are accepted, how many are rejected and for what reasons. Then look at the popularity of different spamdyke features and specifically the popularity of different DNS RBLs. Use all that to find out what percentage of rejected connections could avoid the DNS queries due to local tests. Along those lines, is it possible, or can it be possible, to have spamdyke's logs indicate which DNS RBL caused a message to be rejected? I'm assuming that once a reason for rejection is found, IE, the IP is listed in a particular RBL, further tests against other RBL's in the list are not performed? Knowing, statistically, which ones have a higher rejection rate, and queuing those first in the list of RBLS might save some time. Or course, multiple RBLS could reject the same message, and the one first in line would have the higher percentage, but this would give us a way to move them around and check the results... Just a thought from a newbie to spamdyke. BTW, I LOVE Spamdyke! What a difference it has made in my system's ability to filter spam and save resources! It's a God send! Mike -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Compile Error at 3.1.7 version
Thorsten Puzich wrote: Hello, I get this message, when I run ./configure. checking for __bind in -lsocket... no checking for inet_ntoa in -lnsl... yes checking whether anonymous inner functions are supported by default... no checking whether anonymous inner functions are supported with -fnested- functions... no checking whether anonymous inner functions are supported with - ftrampolines... no configure: error: Unable to compile without anonymous inner function support. What I have to do against this error? Thanks Thorsten Looks like you're missing anonymouns inner function support. I've no idea what that means. ;) What platform/compiler versions are you using? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Allow trusted relays from dynamic ips
Christian Aust wrote: Hi all, I'm using the latest release of spamdyke, and it's working great - thanks a lot. Now I'd like to have my home server relay it's mail through the main mail system. Spamdyke blocks the connecton with DENIED_IP_IN_CC_RDNS, because the home system certainly connects using a non-static IP which happens to have the ip in it's RDNS name. spamdyke is working perfectly and is doing what it has been told. But how could I allow my satellite server to actually send mail through this relay? If I could instruct spamdyke to check the IP against some given dyndns name (and allow if the IPs match) it would be all right, but AFAIK spamdyke doesn't offer such option. Or, does it? Any other ideas? BTW: I'm running postfix on the satellite and (obviously) qmail on the main server. Best regards, Christian Configure your satellite server to authenticate (and probably use TLS too so your authentication password isn't sent in the clear), then spamdyke will bypass all filters. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Compile Error at 3.1.7 version
What does your config.log have to say? Thorsten Puzich wrote: Hi Eric, I am using gentoo linux with this gcc version gcc version 3.4.6 (Gentoo 3.4.6-r2, ssp-3.4.6-1.0, pie-8.7.10) The old spamdyke 3.0 version compiles without any errors. Thanks Thorsten Am 17.04.2008 um 21:57 schrieb Eric Shubert: Thorsten Puzich wrote: Hello, I get this message, when I run ./configure. checking for __bind in -lsocket... no checking for inet_ntoa in -lnsl... yes checking whether anonymous inner functions are supported by default... no checking whether anonymous inner functions are supported with -fnested- functions... no checking whether anonymous inner functions are supported with - ftrampolines... no configure: error: Unable to compile without anonymous inner function support. What I have to do against this error? Thanks Thorsten Looks like you're missing anonymouns inner function support. I've no idea what that means. ;) What platform/compiler versions are you using? -- -Eric 'shubes' -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Allow trusted relays from dynamic ips
Are you simply talking about a right-hand whitelist? That could be useful in some situations. For instance, I recently came across a mailer who was being rejected due to DENIED_RDNS_RESOLVE, so I whitelisted the IP (instead of turning off that check). I would rather whitelist the domain name though, in case they change their server's IP address (which I figure is a fair chance of happening given that it's presently not quite correct). I don't think this should apply to relays (non-local mail) though. Am I missing something here? Sam Clippinger wrote: SMTP AUTH is definitely the best option, if you can configure postfix to perform it for outbound email. I don't use DynDNS myself -- what would be required to support it? Would spamdyke need to find the IP address(es) of a (list of) DynDNS name(s), then add those IP address(es) to the whitelist? If that's all it would take, I don't think that would be very hard. -- Sam Clippinger Christian Aust wrote: Hi all, I'm using the latest release of spamdyke, and it's working great - thanks a lot. Now I'd like to have my home server relay it's mail through the main mail system. Spamdyke blocks the connecton with DENIED_IP_IN_CC_RDNS, because the home system certainly connects using a non-static IP which happens to have the ip in it's RDNS name. spamdyke is working perfectly and is doing what it has been told. But how could I allow my satellite server to actually send mail through this relay? If I could instruct spamdyke to check the IP against some given dyndns name (and allow if the IPs match) it would be all right, but AFAIK spamdyke doesn't offer such option. Or, does it? Any other ideas? BTW: I'm running postfix on the satellite and (obviously) qmail on the main server. Best regards, Christian -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] problems with DENIED_IP_IN_CC_RDNS
Sam Clippinger wrote: Other connections are not being blocked because their rDNS names don't end in country codes. Instead, they use three-character TLDs like .com and .net. If you want to block those connections as well, use the ip-in-rdns-keyword-file option and put .com and .net in the keyword file. That would match the string anywhere in the rdns string though, not only at the end. Might this be a(nother) reason to implement regex matching? (e.g. \.com$) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] problems with DENIED_IP_IN_CC_RDNS
I see. I still think that regex's are more intuitive/flexible though. ;) Sam Clippinger wrote: If the entry starts with a dot, it will only match the end of the rDNS name. If there is no dot, it will match anywhere in the name. -- Sam Clippinger Eric Shubert wrote: Sam Clippinger wrote: Other connections are not being blocked because their rDNS names don't end in country codes. Instead, they use three-character TLDs like .com and .net. If you want to block those connections as well, use the ip-in-rdns-keyword-file option and put .com and .net in the keyword file. That would match the string anywhere in the rdns string though, not only at the end. Might this be a(nother) reason to implement regex matching? (e.g. \.com$) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] problems with DENIED_IP_IN_CC_RDNS
Sam Clippinger wrote: spamdyke looks for the IP address in many different formats. If the IP address is 11.22.33.44, it looks for: 11.22.33.44 011.022.033.044 11.022.033.044 (new in version 4.0.0) 11.22.033.044 (new in version 4.0.0) 11.22.33.044 (new in version 4.0.0) 44.33.22.11 44.11.22.33 33.22.11.44 44.33.1122 3344.11.22 11.22.8492 (last two octets converted to long integer) 11223344 011022033044 11022033044 1122033044 112233044 44332211 044033022011 185999660 (entire IP converted to long integer) 0b16212c (entire IP converted to hex digits) Basically, these are all the different formats I've seen in real life. As people report new ones, I add them too. Here's another one for you Sam: 04-16 13:01:22 DENIED_RDNS_RESOLVE from: [EMAIL PROTECTED] to: [EMAIL PROTECTED] origin_ip: 65.182.166.90 origin_rdns: ihsystem-65-182-166-90.pugmarks.net auth: (unknown) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] problems with DENIED_IP_IN_CC_RDNS
That makes sense, but it's not what I read at http://www.spamdyke.org/documentation/README.html#RDNS I don't see anything there about looking up a corresponding DNS A record. Is the documentation perhaps out of date? (or am I losing it?) ;) Do we perhaps need 2 parameter/rules? One for when the rDNS record does not contain an IP address, and another for when there is no DNS A record for the address that's found? Sam Clippinger wrote: Your example was not rejected by the ip-in-rdns-keyword-file filter. It was rejected by the reject-unresolvable-rdns filter because the rDNS name does not resolve to an IP address (a DNS A record). In other words, ping ihsystem-65-182-166-90.pugmarks.net will fail with unknown host. -- Sam Clippinger Eric Shubert wrote: I don't understand (after having read the documentation) why the example I showed was rejected then. Please explain. Sam Clippinger wrote: Sorry, I should have mentioned that the dots in the formats I listed can actually be any non-alphanumeric character (dashes, underscores, etc). -- Sam Clippinger Eric Shubert wrote: Sam Clippinger wrote: spamdyke looks for the IP address in many different formats. If the IP address is 11.22.33.44, it looks for: 11.22.33.44 011.022.033.044 11.022.033.044 (new in version 4.0.0) 11.22.033.044 (new in version 4.0.0) 11.22.33.044 (new in version 4.0.0) 44.33.22.11 44.11.22.33 33.22.11.44 44.33.1122 3344.11.22 11.22.8492 (last two octets converted to long integer) 11223344 011022033044 11022033044 1122033044 112233044 44332211 044033022011 185999660 (entire IP converted to long integer) 0b16212c (entire IP converted to hex digits) Basically, these are all the different formats I've seen in real life. As people report new ones, I add them too. Here's another one for you Sam: 04-16 13:01:22 DENIED_RDNS_RESOLVE from: [EMAIL PROTECTED] to: [EMAIL PROTECTED] origin_ip: 65.182.166.90 origin_rdns: ihsystem-65-182-166-90.pugmarks.net auth: (unknown) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
[spamdyke-users] Timeout problem
I had a problem receiving a particular email message. It would always send the same amount of data, then timeout. The same amount of data was sent/received with timeouts of 60 and 180 seconds. I logged the message (great little feature of spamdyke btw), and the end part of the message log always shows: HR align=left SIZE=1 color=black div align=leftfont face=arial size=114072172/font/div/td/tr/TBODY/TABLE /BODY/HTML FF 04/22/2008 17:11:13 . QUIT FF 04/22/2008 17:11:13 421 Timeout. Talk faster next time. XX 04/22/2008 17:11:33 250 ok 1208909493 qp 11949 221 doris.shubes.net - Welcome to Qmail Toaster Ver. 1.3 SMTP Server 04/22/2008 17:11:33 CLOSED Here's the smtp log for the successful receipt (with no spamdyke): 04-22 17:21:13 tcpserver: pid 12162 from 208.46.47.130 04-22 17:21:13 tcpserver: ok 12162 doris:192.168.71.11:25 :208.46.47.130::51303 04-22 17:21:13 CHKUSER accepted sender: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 rcpt : sender accepted 04-22 17:21:13 CHKUSER accepted rcpt: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 rcpt [EMAIL PROTECTED] : found existing recipient 04-22 17:21:34 simscan:[12162]:CLEAN (-6.20/99.00):20.2626s:April Edition of MySurvey.com Opinion Matters:208.46.47.130:[EMAIL PROTECTED]:[EMAIL PROTECTED]: 04-22 17:21:34 tcpserver: end 12162 status 0 After receiving the entire message, I see this portion that was received after the part logged by spamdyke: IMG SRC=https://www.mysurvey.com/gems/gems_open_tracking.cfm?indid=14072172cmpid=1105r=1720290rundate=22-APR-2008+11%3a52%3a55z=67129618CF0844A786F0E0A6C20C49CDborder=0; width=1 height=1 --=_Layout_Part_DC7E1BB5_1105_4DB3_BAE3_2A6208EB099A-- Any idea why this would timeout (consistently, like clockwork) with spamdyke, but not without it? This message timed out all day long with spamdyke, but was received successfully on the first attempt without spamdyke. Did spamdyke somehow choke on the last bit? FWIW, it appears that the entire email was a bit hosed, as the html did not render properly in the client view (mac mail) once the entire message was received. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] problems with DENIED_IP_IN_CC_RDNS
Sam Clippinger wrote: You're reading the correct section. The third and fourth paragraphs describe reject-unresolvable-rdns, which is the filter that was triggered in your example. The text doesn't actually use the term A record, instead saying that spamdyke attempts to get an IP address from the name. When I wrote it, I was trying to limit my use of jargon as much as possible. I guess I should rewrite it if it's so unclear. It appears clearer to me now, but I think it could read a little better. This test only attempts to get at least one IP address from the name. It does not require the rDNS name's IP address to match the remote server's IP address. might be replaced with This is done by using the rDNS name to lookup a corresponding IP address. It does not require the corresponding address to be the same as the remote server's IP address, only that the rDNS name correspond to an IP address (or more specifically, a type A DNS record) of some sort. Paragraphs five through ten describe ip-in-rdns-keyword-file and the last paragraph describes reject-ip-in-cc-rdns. I think I could make those read a bit better. Let me know if you'd like me to take a stab at it and we can work it out off list. The two rules you're wanting are already there -- reject-unresolvable-rdns and ip-in-rdns-keyword-file. The former only checks for an A record from the rDNS name. The latter checks for the IP address in the rDNS, plus a keyword from the file. I see that now. I think I may have been having a bit of a brain fart yesterday. ;) Thanks for clearing this up for me. -- Sam Clippinger Eric Shubert wrote: That makes sense, but it's not what I read at http://www.spamdyke.org/documentation/README.html#RDNS I don't see anything there about looking up a corresponding DNS A record. Is the documentation perhaps out of date? (or am I losing it?) ;) Do we perhaps need 2 parameter/rules? One for when the rDNS record does not contain an IP address, and another for when there is no DNS A record for the address that's found? Sam Clippinger wrote: Your example was not rejected by the ip-in-rdns-keyword-file filter. It was rejected by the reject-unresolvable-rdns filter because the rDNS name does not resolve to an IP address (a DNS A record). In other words, ping ihsystem-65-182-166-90.pugmarks.net will fail with unknown host. -- Sam Clippinger Eric Shubert wrote: I don't understand (after having read the documentation) why the example I showed was rejected then. Please explain. Sam Clippinger wrote: Sorry, I should have mentioned that the dots in the formats I listed can actually be any non-alphanumeric character (dashes, underscores, etc). -- Sam Clippinger Eric Shubert wrote: Sam Clippinger wrote: spamdyke looks for the IP address in many different formats. If the IP address is 11.22.33.44, it looks for: 11.22.33.44 011.022.033.044 11.022.033.044 (new in version 4.0.0) 11.22.033.044 (new in version 4.0.0) 11.22.33.044 (new in version 4.0.0) 44.33.22.11 44.11.22.33 33.22.11.44 44.33.1122 3344.11.22 11.22.8492 (last two octets converted to long integer) 11223344 011022033044 11022033044 1122033044 112233044 44332211 044033022011 185999660 (entire IP converted to long integer) 0b16212c (entire IP converted to hex digits) Basically, these are all the different formats I've seen in real life. As people report new ones, I add them too. Here's another one for you Sam: 04-16 13:01:22 DENIED_RDNS_RESOLVE from: [EMAIL PROTECTED] to: [EMAIL PROTECTED] origin_ip: 65.182.166.90 origin_rdns: ihsystem-65-182-166-90.pugmarks.net auth: (unknown) ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting wishes
There's possible, and there's trivial. I vote trivial (good idea!). :) Sam Clippinger wrote: ALLOWED_GRAYLISTED could be useful if graylisting isn't active for all domains. It would mean that the graylisting filter had checked for the existence of a graylist file for that connection (and found one). I agree it should be possible to match an ALLOWED with a previous DENIED_GRAYLISTED but that could involve searching log files from multiple days if the remote server doesn't attempt redelivery very quickly. -- Sam Clippinger Michael Colvin wrote: Doesn't it already log DENIED GREYLISTED when it greylists an address, then when it is sent again, and passes the greylist test, it logs ALLOWED... Doesn't that already identify greylisted e-mails? Or, are we talking about logging the fact that e-mails are allowed AND have already been greylisted? Which, if you greylist all domains, would be every e-mail, right? The ALLOWED_WHITELISTED_* items might be useful, but I don't see where logging allowed greylisted e-mails makes sense... In fact, Allowed Greylist seems kind of contradictory to me... :-) Just my .02, which, with the state of the dollar, is worth even less today than last week. :-) Michael J. Colvin NorCal Internet Services www.norcalisp.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of BC Sent: Wednesday, April 23, 2008 1:32 PM To: spamdyke-users@spamdyke.org Subject: Re: [spamdyke-users] Greylisting wishes On 4/23/2008 [EMAIL PROTECTED] wrote: I could do that if it would be useful. Now is the time for changes like this, since version 4.0 won't be backwards compatible anyway. What about changing the log message for other reasons too? For example, ALLOWED_WHITELISTED_IP, ALLOWED_WHITELISTED_SENDER, etc. I'd like to see that sort of addition to the logging, too. Thanks, Bucky -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting wishes
I think I sorta like both. Sam Clippinger wrote: OK, I guess I've been working on version 4.0.0 for too long now because I didn't realize I'd already implemented this feature (until I tried to add it again). However, I didn't do it quite the way we described in this thread; instead of changing the ALLOWED messages, I added a new log level that will print out extra messages. (In fact, the entire log level system has been revisited and reorganized). When the logging level is verbose or higher, messages like these will be produced: FILTER_RDNS_MISSING ip: 11.22.33.44 FILTER_RDNS_BLACKLIST ip: 11.22.33.44 rdns: 11-22-33-44.example.com file: /var/qmail/spamdyke/rdns_blacklist.txt(31) FILTER_RBL_MATCH ip: 11.22.33.44 rbl: foorbl.example.com FILTER_GRAYLISTED sender: [EMAIL PROTECTED] recipient: [EMAIL PROTECTED] path: /var/qmail/spamdyke/graylist.d/example.com/user/spamdomain.com/spammer FILTER_WHITELIST_IP ip: 11.22.33.44 file: /var/qmail/spamdyke/whitelist_ip.txt(7) ...and so on. Any filter that triggers either an acceptance or a rejection will produce a FILTER log message. Filters that only examine the connection (but aren't triggered) won't produce any output (unless the log level is increased to debug or higher). I chose this approach because it provides more information than just the matching filter; it gives the file and line numbers, the directory paths, etc. Because it requires setting the log level higher, it can be enabled when someone wants to collect the data for analysis or turned off if it is not wanted. Does that sound sufficient or should I remove it and change the ALLOWED messages instead? -- Sam Clippinger Sam Clippinger wrote: ALLOWED_GRAYLISTED could be useful if graylisting isn't active for all domains. It would mean that the graylisting filter had checked for the existence of a graylist file for that connection (and found one). I agree it should be possible to match an ALLOWED with a previous DENIED_GRAYLISTED but that could involve searching log files from multiple days if the remote server doesn't attempt redelivery very quickly. -- Sam Clippinger Michael Colvin wrote: Doesn't it already log DENIED GREYLISTED when it greylists an address, then when it is sent again, and passes the greylist test, it logs ALLOWED... Doesn't that already identify greylisted e-mails? Or, are we talking about logging the fact that e-mails are allowed AND have already been greylisted? Which, if you greylist all domains, would be every e-mail, right? The ALLOWED_WHITELISTED_* items might be useful, but I don't see where logging allowed greylisted e-mails makes sense... In fact, Allowed Greylist seems kind of contradictory to me... :-) Just my .02, which, with the state of the dollar, is worth even less today than last week. :-) Michael J. Colvin NorCal Internet Services www.norcalisp.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of BC Sent: Wednesday, April 23, 2008 1:32 PM To: spamdyke-users@spamdyke.org Subject: Re: [spamdyke-users] Greylisting wishes On 4/23/2008 [EMAIL PROTECTED] wrote: I could do that if it would be useful. Now is the time for changes like this, since version 4.0 won't be backwards compatible anyway. What about changing the log message for other reasons too? For example, ALLOWED_WHITELISTED_IP, ALLOWED_WHITELISTED_SENDER, etc. I'd like to see that sort of addition to the logging, too. Thanks, Bucky -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Timeout problem
That's interesting, Paulo. I have timeoutsmtpd at 600, and nothing specified for idle-timeout-secs. Sam's having a look at a couple of my logs. I'll be glad to try this out if Sam gives me the word (I don't want to mess up his debugging efforts). I wonder if idle-timeout-secs is somehow not being initialized/defaulted properly. Thanks for the input Paulo. Paulo Henrique wrote: I had a problem like this and decided putting the timeout from qmail less than the timeout from spamdyke, see: cat /var/qmail/control/timeoutsmtpd 240 grep idle-timeout-secs /var/qmail/control/spamdyke/spamdyke.conf idle-timeout-secs = 300 After that never had problem with the repetition of messages. 2008/4/22 Eric Shubert [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: I had a problem receiving a particular email message. It would always send the same amount of data, then timeout. The same amount of data was sent/received with timeouts of 60 and 180 seconds. I logged the message (great little feature of spamdyke btw), and the end part of the message log always shows: HR align=left SIZE=1 color=black div align=leftfont face=arial size=114072172/font/div/td/tr/TBODY/TABLE /BODY/HTML FF 04/22/2008 17:11:13 . QUIT FF 04/22/2008 17:11:13 421 Timeout. Talk faster next time. XX 04/22/2008 17:11:33 250 ok 1208909493 qp 11949 221 doris.shubes.net http://doris.shubes.net - Welcome to Qmail Toaster Ver. 1.3 SMTP Server 04/22/2008 17:11:33 CLOSED Here's the smtp log for the successful receipt (with no spamdyke): 04-22 17:21:13 tcpserver: pid 12162 from 208.46.47.130 http://208.46.47.130 04-22 17:21:13 tcpserver: ok 12162 doris:192.168.71.11:25 http://192.168.71.11:25 :208.46.47.130::51303 04-22 17:21:13 CHKUSER accepted sender: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 http://208.46.47.130 rcpt : sender accepted 04-22 17:21:13 CHKUSER accepted rcpt: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 http://208.46.47.130 rcpt [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] : found existing recipient 04-22 17:21:34 simscan:[12162]:CLEAN (-6.20/99.00):20.2626s:April Edition of MySurvey.com Opinion Matters:208.46.47.130:[EMAIL PROTECTED]:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: 04-22 17:21:34 tcpserver: end 12162 status 0 After receiving the entire message, I see this portion that was received after the part logged by spamdyke: IMG SRC=https://www.mysurvey.com/gems/gems_open_tracking.cfm?indid=14072172cmpid=1105r=1720290rundate=22-APR-2008+11%3a52%3a55z=67129618CF0844A786F0E0A6C20C49CD https://www.mysurvey.com/gems/gems_open_tracking.cfm?indid=14072172cmpid=1105r=1720290rundate=22-APR-2008+11%3a52%3a55z=67129618CF0844A786F0E0A6C20C49CDborder=0 width=1 height=1 --=_Layout_Part_DC7E1BB5_1105_4DB3_BAE3_2A6208EB099A-- Any idea why this would timeout (consistently, like clockwork) with spamdyke, but not without it? This message timed out all day long with spamdyke, but was received successfully on the first attempt without spamdyke. Did spamdyke somehow choke on the last bit? FWIW, it appears that the entire email was a bit hosed, as the html did not render properly in the client view (mac mail) once the entire message was received. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org mailto:spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- Paulo Henrique Fonseca [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Timeout problem
I was wrong - I *do* have idle-timeout-secs specified as 60. I'll go ahead and bump it up just above my timoutsmtpd value, and we'll see what happens. Sam Clippinger wrote: I would be very interested to know if that solves your problem, Eric. I can't see why it would, but since I don't (yet) understand what's wrong, I can't rule anything out. :) Looking through the logs you gave me, I can't see why the timeout is being triggered at all. The remote server is sending data constantly, so the idle timer should be reset multiple times per second. I think it may have something to do with the exact composition of the message -- it's possible that there's a bug in the way spamdyke manipulates its buffers to hold and move data (version 3.1.6 fixed a problem like this). At the moment, I'm trying to reconstruct the message that's triggering this bug on Eric's server so I can reproduce this error myself. I haven't had any success triggering this bug by using just any large message. -- Sam Clippinger Eric Shubert wrote: That's interesting, Paulo. I have timeoutsmtpd at 600, and nothing specified for idle-timeout-secs. Sam's having a look at a couple of my logs. I'll be glad to try this out if Sam gives me the word (I don't want to mess up his debugging efforts). I wonder if idle-timeout-secs is somehow not being initialized/defaulted properly. Thanks for the input Paulo. Paulo Henrique wrote: I had a problem like this and decided putting the timeout from qmail less than the timeout from spamdyke, see: cat /var/qmail/control/timeoutsmtpd 240 grep idle-timeout-secs /var/qmail/control/spamdyke/spamdyke.conf idle-timeout-secs = 300 After that never had problem with the repetition of messages. 2008/4/22 Eric Shubert [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: I had a problem receiving a particular email message. It would always send the same amount of data, then timeout. The same amount of data was sent/received with timeouts of 60 and 180 seconds. I logged the message (great little feature of spamdyke btw), and the end part of the message log always shows: HR align=left SIZE=1 color=black div align=leftfont face=arial size=114072172/font/div/td/tr/TBODY/TABLE /BODY/HTML FF 04/22/2008 17:11:13 . QUIT FF 04/22/2008 17:11:13 421 Timeout. Talk faster next time. XX 04/22/2008 17:11:33 250 ok 1208909493 qp 11949 221 doris.shubes.net http://doris.shubes.net - Welcome to Qmail Toaster Ver. 1.3 SMTP Server 04/22/2008 17:11:33 CLOSED Here's the smtp log for the successful receipt (with no spamdyke): 04-22 17:21:13 tcpserver: pid 12162 from 208.46.47.130 http://208.46.47.130 04-22 17:21:13 tcpserver: ok 12162 doris:192.168.71.11:25 http://192.168.71.11:25 :208.46.47.130::51303 04-22 17:21:13 CHKUSER accepted sender: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 http://208.46.47.130 rcpt : sender accepted 04-22 17:21:13 CHKUSER accepted rcpt: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 http://208.46.47.130 rcpt [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] : found existing recipient 04-22 17:21:34 simscan:[12162]:CLEAN (-6.20/99.00):20.2626s:April Edition of MySurvey.com Opinion Matters:208.46.47.130:[EMAIL PROTECTED]:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: 04-22 17:21:34 tcpserver: end 12162 status 0 After receiving the entire message, I see this portion that was received after the part logged by spamdyke: IMG SRC=https://www.mysurvey.com/gems/gems_open_tracking.cfm?indid=14072172cmpid=1105r=1720290rundate=22-APR-2008+11%3a52%3a55z=67129618CF0844A786F0E0A6C20C49CD https://www.mysurvey.com/gems/gems_open_tracking.cfm?indid=14072172cmpid=1105r=1720290rundate=22-APR-2008+11%3a52%3a55z=67129618CF0844A786F0E0A6C20C49CDborder=0 width=1 height=1 --=_Layout_Part_DC7E1BB5_1105_4DB3_BAE3_2A6208EB099A-- Any idea why this would timeout (consistently, like clockwork) with spamdyke, but not without it? This message timed out all day long with spamdyke, but was received successfully on the first attempt without spamdyke. Did spamdyke somehow choke on the last bit? FWIW, it appears that the entire email was a bit hosed, as the html did not render properly in the client view (mac mail) once the entire message was received. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org mailto:spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo
Re: [spamdyke-users] Timeout problem
The clamav problem shouldn't be coming into play, as I had already upgraded to clamav-0.92.1 before installing spamdyke. Thanks for the reminder about that though. FWIW, the server in question is a PII/266/512 (try not to laugh too hard). It's load average is typically 0.2 or less though. Sam Clippinger wrote: The issue where timed-out messages are delivered anyway will be fixed in version 4.0.0. I don't see how ClamAV could be causing Eric's timeouts but again, since I don't (yet) understand what's happening, it's worth a shot. Keeping ClamAV up to date is always a good idea, whether any problems are occurring or not. Generally speaking, a slow/unresponsive qmail (or other child process) can cause an idle timeout in spamdyke 3.1.7 -- I've fixed this in the next version. -- Sam Clippinger Bruce Schreiber wrote: Michael, I had the exact same symptom with multiple users. The problem turned out to be in ClamAV. There is a DOS exploit in ClamAV that is solved with an upgrade to 0.91 or later. (see http://xforce.iss.net/xforce/xfdb/35367) Upgrading ClamAV solved the problem for the most part. I agree that the symptom is disturbing. If the mail client is being sent a message indicating that the message failed, then it should not be sent by Qmail. I believe this is a Spamdyke bug. Spamdyke is terminating the client session, but is failing to stop Qmail from sending the email. Outlook exacerbates this problem by automatically retrying the failed message, without notifying the user. I had one customer complain that a message was sent 170 times. Customers eyes glaze over when you try to explain why it happened. Sam, I would appreciate your thoughts on this. Bruce Michael Colvin wrote: Doing this, kind of negates the need for doing it in SpamDyke, except for maybe a Backup in case Qmail doesn't for some reason. I think the problem is, some people don't have a timeoutsmtpd file. I had a Stock Qmailrocks install that did not have it, and apparently, the Default value used by Qmail if that file is missing is 1200 seconds (20 minutes), which of course is kind of ridiculous. So, with even a modest value in SpamDyke of 300 seconds, SpamDyke would occassionally timeout a connection, and in some cases, I think because of the way SpamDyke disconnected the session, the sending server didn't realize the message had been sent. I belive it is discusses in this thread: http://www.mail-archive.com/spamdyke-users@spamdyke.org/msg00746.html **Michael J. Colvin** **NorCal Internet Services** **//www.norcalisp.com// http://www.norcalisp.com/** http://www.norcalisp.com/ *From:* [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] *On Behalf Of *Paulo Henrique *Sent:* Sunday, April 27, 2008 6:18 PM *To:* spamdyke users *Subject:* Re: [spamdyke-users] Timeout problem I had a problem like this and decided putting the timeout from qmail less than the timeout from spamdyke, see: cat /var/qmail/control/timeoutsmtpd 240 grep idle-timeout-secs /var/qmail/control/spamdyke/spamdyke.conf idle-timeout-secs = 300 After that never had problem with the repetition of messages. 2008/4/22 Eric Shubert [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: I had a problem receiving a particular email message. It would always send the same amount of data, then timeout. The same amount of data was sent/received with timeouts of 60 and 180 seconds. I logged the message (great little feature of spamdyke btw), and the end part of the message log always shows: HR align=left SIZE=1 color=black div align=leftfont face=arial size=114072172/font/div/td/tr/TBODY/TABLE /BODY/HTML FF 04/22/2008 17:11:13 . QUIT FF 04/22/2008 17:11:13 421 Timeout. Talk faster next time. XX 04/22/2008 17:11:33 250 ok 1208909493 qp 11949 221 doris.shubes.net http://doris.shubes.net - Welcome to Qmail Toaster Ver. 1.3 SMTP Server 04/22/2008 17:11:33 CLOSED Here's the smtp log for the successful receipt (with no spamdyke): 04-22 17:21:13 tcpserver: pid 12162 from 208.46.47.130 http://208.46.47.130 04-22 17:21:13 tcpserver: ok 12162 doris:192.168.71.11:25 http://192.168.71.11:25 :208.46.47.130::51303 04-22 17:21:13 CHKUSER accepted sender: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 http://208.46.47.130 rcpt : sender accepted 04-22 17:21:13 CHKUSER accepted rcpt: from [EMAIL PROTECTED]:: remote rapport.mysurvey.com:unknown:208.46.47.130 http://208.46.47.130 rcpt [EMAIL
Re: [spamdyke-users] DNSRBL question
The old RBLSMTPD is doing the lookup before passing it on to spamdyke, so spamdyke is never receiving it. You need to remove $RBLSMTPD $BLACLISTS to disable the toaster's stock blacklist processing. You didn't use qtp-install-spamdyke, did you? It would have modified your run file to look like this: #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` SPAMDYKE=/usr/local/bin/spamdyke SPAMDYKE_CONF=/etc/spamdyke/spamdyke.conf SMTPD=/var/qmail/bin/qmail-smtpd TCP_CDB=/etc/tcprules.d/tcp.smtp.cdb HOSTNAME=`hostname` VCHKPW=/home/vpopmail/bin/vchkpw REQUIRE_AUTH=0 exec /usr/bin/softlimit -m 2000 \ /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c $MAXSMTPD \ -u $QMAILDUID -g $NOFILESGID 0 smtp \ $SPAMDYKE --config-file $SPAMDYKE_CONF \ $SMTPD $VCHKPW /bin/true 21 I think that's a little cleaner than burying spamdyke in $SMTPD as you have done. If you use the above run script, be sure to change /etc/spamdyke to /etc/mail, or move your spamdyke.conf file appropriately. slamp slamp wrote: I have a question. I have the line below in my config. check-dnsrbl=zen.spamhaus.org So spamdyke should check if the sender is listed correct? and it should never need to pass the traffic to qmail? My observation so far seems that spamdyke is not doing this and my qmail install (qmailtoaster) is still doing the dns rbl checking. My qmail smtp log says this: 2008-05-03 10:27:50.146011500 rblsmtpd: 201.12.53.18 pid 27047: 451 http://www.spamhaus.org/query/bl?ip=201.12.53.18 And nothing in the corresponding spamdyke maillog. log-level=2 smtp run file. #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` BLACKLIST=`cat /var/qmail/control/blacklists` SMTPD=/usr/bin/spamdyke -f /etc/mail/spamdyke.conf /var/qmail/bin/qmail-smtpd TCP_CDB=/etc/tcprules.d/tcp.smtp.cdb RBLSMTPD=/usr/bin/rblsmtpd HOSTNAME=`hostname` VCHKPW=/home/vpopmail/bin/vchkpw REQUIRE_AUTH=0 exec /usr/bin/softlimit -m 2000 \ /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c $MAXSMTPD \ -u $QMAILDUID -g $NOFILESGID 0 smtp \ $RBLSMTPD $BLACKLIST $SMTPD $VCHKPW /bin/true 21 ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] yet another wishlist... :-)
Well said. It wouldn't be spamDYKE at that point. ;) Bgs wrote: Spamdyke is an smtp level filtering system while virus filtering is at the data level. Absolutely different by design. Spamdyke is fast because it does not bother to handle data. If you add virus filtering to it, it would be just-another-virus-scanner-with-dns-checks. It would loose most of what it makes valuable. to be able to virus scan you need to queue the data, which takes hdd space, IO, queuing system, etc. Right now data is just passed through. With tls you would loose overview anyway so part of the mails cannot be filtered. Bye Bgs Olivier Mueller wrote: On Fri, 2008-05-16 at 15:39 +0200, Marcin Orlowski wrote: Sam Clippinger wrote: I'd love to be able to do spam and virus scanning within spamdyke, But what for? There's couple of tools you can use to scan (for whatever you want) incoming mails before they go to the user mailbox and drop mails when needed. Absolutely pointless feature to be added to spamdyke Yes, but not always on SMTP-level, and IMHO it's better there since the sender (if he's in the 3-4% of non-spams) will get an error message from his smtp server in case of problems. Otherwise it will be silently dropped, and it's unpractical to debug issues... regards, Olivier -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] unable to do make at version 4.0.3
nightduke wrote: -bash-3.1# ./configure checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for stdint.h... (cached) yes checking sys/inttypes.h usability... no checking sys/inttypes.h presence... no checking for sys/inttypes.h... no checking whether time.h and sys/time.h may both be included... yes checking for int16_t... yes checking for int32_t... yes checking for int64_t... yes checking for uint16_t... yes checking for uint32_t... yes checking for uint64_t... yes checking for dirent.h that defines DIR... yes checking for library containing opendir... none required checking for struct dirent.d_type... yes checking whether DT_WHT is declared... yes checking whether S_IFWHT is declared... no checking whether INADDR_LOOPBACK is declared... yes checking whether to include debugging symbols (for gdb)... no checking for strip... strip spamdyke checking whether to include excessive debugging output... no checking whether to include some debugging output... yes checking whether to include configuration tests... yes checking if openssl/ssl.h will include without additional include directories... no configure: Adding /usr/kerberos/include to the include file search path checking Checking if openssl/ssl.h will include correctly... no configure: Unable to include openssl/ssl.h (required by OpenSSL), TLS support di sabled checking for library containing inet_aton... none required checking for library containing bind... none required checking for library containing inet_ntoa... none required checking for library containing getopt_long... none required checking whether anonymous inner functions are supported by default... yes checking whether struct option is defined in getopt.h... yes checking whether pid_t is an unsigned int or an unsigned long... unsigned int checking whether uid_t is an unsigned int or an unsigned long... unsigned int checking whether gid_t is an unsigned int or an unsigned long... unsigned int checking whether long long ints are supported in a test program... yes checking whether printf()/scanf() uses %lld for 64-bit integers... yes checking whether __func__ is available... yes configure: creating ./config.status config.status: creating Makefile config.status: creating config.h config.status: config.h is unchanged -bash-3.1# make -bash: make: command not found -bash-3.1# -bash-3.1# pwd /root/spamdyke/spamdyke-4.0.3/spamdyke How can i fix this little problem? Thanks Nightduke Where's your make? Either make's not installed: # rpm -ql make or /usr/bin isn't in your path: # echo $PATH -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spamassassin and spamdyke
If you're using qmailtoaster, you can submit to port 587, which always authenticates, and I believe does not invoke spamassassin. I'm not 100% positive about this though. nightduke wrote: Yes that's my idea whitelist anyone that authenticates to my qmail server and bypass spamassassin... It's possible? Thanks Nightduke 2008/8/19 David Stiller [EMAIL PROTECTED]: Oh, sorry... missed the subject ;) I don't scan outgoing mails with spamassassin^^ nightduke schrieb: Hi i wish to know if can be done bypass spamdyke if spamdyke accepts smtp auth connection? I would like to trust on customer who sign on correctly at smtp and then starts the delivery... It's possible to do that? Thanks Nightduke ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- Technischer Support/ Hotline BLACKBIT neue Medien GmbH | BLACKBIT neue Werbung GmbH Ernst-Ruhstrat-Str. 6 - D-37079 Göttingen Geschäftsführer: Stefano Viani Registergericht: Amtsgericht Göttingen, HRB 3222 Umsatzsteueridentifikationsnummer (§ 27a UstG): DE 813 114 917 Tel: +49-551-50675-50 - Fax: +49-551-50675-20 E-Mail: [EMAIL PROTECTED] Klassische Werbung und Online-Marketing: http://www.blackbit.de Software fuer Online-Marketing: http://www.go-community.de ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spamdyke 4.0.3 not allowing?
Sam Clippinger wrote: Good to hear it's working... I guess there just weren't any good messages being delivered while you were testing filter-level? That's what I'm thinking. I'm still seeing something a little peculiar though. I would expect every smtp session to generate a spamdyke message of one form or another, either a rejection or an allow. This particular server's pretty, so it's sometimes hard to tell. Is this the case, or are there situations where a session might not have a spamdyke message? FWIW, this server is simply a relay for specific domains, and has/does no authentication other than checking rcpthosts and morercpthosts, then forwards the mail based on the .qmail-default record for each domain. Kinda goofy, I know. By the way, setting the filter-level option in the global config file is not really what I had in mind when I created that flag. Since it overrides all other flags, including blacklists, it was really intended for use in configuration directories. Specifically, some of my users have become tired of repeatedly asking me to whitelist their correspondents. Several have asked me to just turn off spam filtering for their accounts. With configuration directories, I can create a file for their address that includes the command filter-level=allow-all (they typically begin to see the wisdom of filtering after a few days). Without that command, their file would have to explicitly disable all enabled filters and would be a pain to create/maintain. By the same token, I wanted to provide an easy way for administrators to require authentication for senders/recipients within specific domains. That is now very easy to accomplish using a configuration directory and filter-level=require-auth. Nice. FWIW, I just found it to be an easy way to turn spamdyke off temporarily, as opposed to changing run files back and forth. :) -- Sam Clippinger Eric Shubert wrote: Eric Shubert wrote: Eric Shubert wrote: I've probably hosed up something in my new .conf file. What I'm seeing is that with filter-level=normal, I'm seeing some rejections (not as many as I'd expect), and NO allow messages. I can confirm that nothing is being allowed from looking at the send queue. With filter-level=allow-all, it's indeed allowing everything. Not exactly what I had in mind though. :( Here's my spamdyke.conf file: filter-level=allow-all max-recipients=50 reject-empty-rdns reject-ip-in-cc-rdns reject-missing-sender-mx reject-unresolvable-rdns log-level=info log-target=stderr idle-timeout-secs=300 ip-blacklist-file=/etc/spamdyke/blacklist_ip rdns-blacklist-file=/etc/spamdyke/blacklist_rdns recipient-blacklist-file=/etc/spamdyke/blacklist_recipients sender-blacklist-file=/etc/spamdyke/blacklist_senders ip-in-rdns-keyword-blacklist-file=/etc/spamdyke/blacklist_keywords ip-whitelist-file=/etc/spamdyke/whitelist_ip rdns-whitelist-file=/etc/spamdyke/whitelist_rdns recipient-whitelist-file=/etc/spamdyke/whitelist_recipients sender-whitelist-file=/etc/spamdyke/whitelist_senders ip-in-rdns-keyword-whitelist-file=/etc/spamdyke/whitelist_keywords dns-blacklist-entry=zen.spamhaus.org dns-blacklist-entry=bl.spamcop.net graylist-level=always-create-dir graylist-dir=/var/spamdyke/graylist graylist-max-secs=1814400 graylist-min-secs=180 local-domains-file=/var/qmail/control/rcpthosts local-domains-file=/var/qmail/control/morercpthosts Note, in the cases where the parameter references a file, the file exists and is empty. Thoughts / suggestions? Ok, so I removed all of the blacklist and whilelist file references, and graylisting, and I'm seeing an allow or 2 coming through. That's good! I'll try adding parameters back in and see if I can pinpoint the culprit. Ok, so there doesn't appear to be a problem any more. After some careful testing, everything appears to be working as it should. As Rosanna Rosannadanna would say, Never mind. ;) ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spamdyke 4.0.3 logging
I think I've taken splogger out of the picture. I have qmail-send messages going to the (proper) /var/log/qmail/send/current file via multilog. The qmail logs look ok now. However, the spamdyke messages are still going to both the smtpd/current log, as well as /var/log/maillog. Any idea how that could be happening? Sam Clippinger wrote: I'm not that familiar with splogger but a quick search gave me several pages that all say it sends its messages to syslog in addition to passing them through stdout/stderr. http://www.ezmlm.org/man/man8/splogger.8.html Of course it's possible this is a spamdyke bug but the way you've described your setup, it sounds like splogger is functioning correctly. -- Sam Clippinger Eric Shubert wrote: Eric Shubert wrote: I've just installed spamdyke 4.0.3 on a somewhat convoluted qmail host, and am seeing some wierdness with logging. The server has logging for qmail-smtp set up in the typical qmail fashion, with logging going to stderr and on to /var/log/qmail/smtpd/current. I have spamdyke configured with log-target=stderr. Logging looks fine in the smtpd log. Now for the weirdness. The qmail-start (and thus qmail-send) is configured to use splogger to send messages to /var/log/maillog. Why, I have no idea. The weird thing is that spamdyke's messages are appearing in /var/log/maillog as well as /var/log/qmail/smtp/current. Any idea how/why this is happening? Could be something in the (mis)configuration that I'm not seeing, but I'm a bit befuddled. So I've moved qmail-send's logging to where it's usually found, at /var/log/qmail/send/current. Spamdyke's log messages are still showing up in /var/log/maillog though, in addition to /var/log/qmail/smtpd/current (where they're supposed to go). I double checked configuration, and I have log-target=stderr. Looking like a bug to me, Sam. ;) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spamdyke 4.0.3 not allowing?
That's pretty much what I figured. I've kept a little closer eye on it this morning (just visual monitoring), and it seems to be rejecting nearly everything properly. Maybe a couple instances where the session ends with status 0 and no message from spamdyke. Would it be possible to add a 'sender disconnect' or some such message, so that there will always be a message from spamdyke for every smtp session that's initiated? Not a big deal, but it'd be nice to be able to account for every connection (if someone were to write a log summary report of some kind). Sam Clippinger wrote: spamdyke won't log anything if a remote client disconnects without identifying a sender or recipient. Prior to version 4.0, it wouldn't log anything if a message was delivered with TLS but that's been fixed. I can't think of any other situation where a delivery (or rejection) would not create a log entry. -- Sam Clippinger Eric Shubert wrote: Sam Clippinger wrote: Good to hear it's working... I guess there just weren't any good messages being delivered while you were testing filter-level? That's what I'm thinking. I'm still seeing something a little peculiar though. I would expect every smtp session to generate a spamdyke message of one form or another, either a rejection or an allow. This particular server's pretty, so it's sometimes hard to tell. Is this the case, or are there situations where a session might not have a spamdyke message? FWIW, this server is simply a relay for specific domains, and has/does no authentication other than checking rcpthosts and morercpthosts, then forwards the mail based on the .qmail-default record for each domain. Kinda goofy, I know. By the way, setting the filter-level option in the global config file is not really what I had in mind when I created that flag. Since it overrides all other flags, including blacklists, it was really intended for use in configuration directories. Specifically, some of my users have become tired of repeatedly asking me to whitelist their correspondents. Several have asked me to just turn off spam filtering for their accounts. With configuration directories, I can create a file for their address that includes the command filter-level=allow-all (they typically begin to see the wisdom of filtering after a few days). Without that command, their file would have to explicitly disable all enabled filters and would be a pain to create/maintain. By the same token, I wanted to provide an easy way for administrators to require authentication for senders/recipients within specific domains. That is now very easy to accomplish using a configuration directory and filter-level=require-auth. Nice. FWIW, I just found it to be an easy way to turn spamdyke off temporarily, as opposed to changing run files back and forth. :) -- Sam Clippinger Eric Shubert wrote: Eric Shubert wrote: Eric Shubert wrote: I've probably hosed up something in my new .conf file. What I'm seeing is that with filter-level=normal, I'm seeing some rejections (not as many as I'd expect), and NO allow messages. I can confirm that nothing is being allowed from looking at the send queue. With filter-level=allow-all, it's indeed allowing everything. Not exactly what I had in mind though. :( Here's my spamdyke.conf file: filter-level=allow-all max-recipients=50 reject-empty-rdns reject-ip-in-cc-rdns reject-missing-sender-mx reject-unresolvable-rdns log-level=info log-target=stderr idle-timeout-secs=300 ip-blacklist-file=/etc/spamdyke/blacklist_ip rdns-blacklist-file=/etc/spamdyke/blacklist_rdns recipient-blacklist-file=/etc/spamdyke/blacklist_recipients sender-blacklist-file=/etc/spamdyke/blacklist_senders ip-in-rdns-keyword-blacklist-file=/etc/spamdyke/blacklist_keywords ip-whitelist-file=/etc/spamdyke/whitelist_ip rdns-whitelist-file=/etc/spamdyke/whitelist_rdns recipient-whitelist-file=/etc/spamdyke/whitelist_recipients sender-whitelist-file=/etc/spamdyke/whitelist_senders ip-in-rdns-keyword-whitelist-file=/etc/spamdyke/whitelist_keywords dns-blacklist-entry=zen.spamhaus.org dns-blacklist-entry=bl.spamcop.net graylist-level=always-create-dir graylist-dir=/var/spamdyke/graylist graylist-max-secs=1814400 graylist-min-secs=180 local-domains-file=/var/qmail/control/rcpthosts local-domains-file=/var/qmail/control/morercpthosts Note, in the cases where the parameter references a file, the file exists and is empty. Thoughts / suggestions? Ok, so I removed all of the blacklist and whilelist file references, and graylisting, and I'm seeing an allow or 2 coming through. That's good! I'll try adding parameters back in and see if I can pinpoint the culprit. Ok, so there doesn't appear to be a problem any more. After some careful testing, everything appears to be working as it should. As Rosanna
Re: [spamdyke-users] Spamdyke and cron jobs...
nightduke wrote: Sep 4 08:00:48 vps spamdyke[5229]: FILTER_SENDER_NO_MX domain: localhost Sep 4 08:00:48 vps spamdyke[5229]: DENIED_SENDER_NO_MX from: [EMAIL PROTECTED] ost to: [EMAIL PROTECTED] origin_ip: 127.0.0.1 origin_rdns: localhost au th: (unknown) Hum i have a few cron jobs after they start send an email with the result but i saw the log and it's denied. How can i fix put in whitelist ip 127.0.0.1 it a good choice? Nightduke Good choice. Another way might be to have cron jobs authenticate. Not sure if that's even possible. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] New version: spamdyke 4.0.4
What's the word on qtp release? It appears that 1.4.5 doesn't have the updated qtp-install-spamdyke. That release was missing the updated qtp-newmodel as well, but did happen to contain qtp-install-rpmforge. Let me know when we can get this straightened out so we can announce the release to the list. Thanks. Sam Clippinger wrote: spamdyke version 4.0.4 is now available: http://www.spamdyke.org/ This version fixes two bugs. The first is a compiler warning on 64-bit Linux systems. Thanks to kjl for reporting this one. The second is a logging bug that was sending log messages to both standard error and the system log. Thanks to Eric Shubert for reporting this one. Version 4.x is NOT backwards compatible with 3.x; be sure to read the documentation before upgrading. Version 4.0.4 is backwards-compatible with version 4.0.3; simply replacing the old binary with the new one should be safe. -- Sam Clippinger -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Graylite and whitelist problems
I think I can field this one. ;) Davide D'AMICO wrote: Hi, I'm using spamdyke and I like it a lot. I encountered two problems: 1) Isn't more useful to graylist senders using their ip address rather than only its email address, like this: /var/db/spamdyke/graylist/domain/rcpt/sender/ip_sender ? Some large (think yahoo, gmail) mailers use server pools. Retries might be sent from a different server, causing a message to be graylisted many times. Personally, I think it'd be ok to use IPs for a type of whitelist after the IP has passed graylisting. After all, once an IP has passed for one domain/sender, wouldn't it pass for all other domain/senders too? However, this adds another level of complexity (a pre- and a passed- gray list, sometimes referred to as a dual key). If this proved to be a good method, a global whitelist service based on the post-key (simply IP address), sort of like RBLSs but RWLs, could be implemented. I don't know if anyone's pursued such a thing or not. Seems feasible to me though. 2) if I include an ip address in a whitelist, I become a relay for that ip address because that ip address bypass ALL other filters? No, because authentication is still required for non-local domains. Spamdyke filters are only bypassed if/when the sender authenticates. Thanks in advance, Davide -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Graylite and whitelist problems
Davide D'AMICO wrote: 2008/9/7 Eric Shubert [EMAIL PROTECTED]: I think I can field this one. ;) Davide D'AMICO wrote: 1) Isn't more useful to graylist senders using their ip address rather than only its email address, like this: /var/db/spamdyke/graylist/domain/rcpt/sender/ip_sender ? Some large (think yahoo, gmail) mailers use server pools. Retries might be sent from a different server, causing a message to be graylisted many times. Personally, I think it'd be ok to use IPs for a type of whitelist after the IP has passed graylisting. After all, once an IP has passed for one domain/sender, wouldn't it pass for all other domain/senders too? However, this adds another level of complexity (a pre- and a passed- gray list, sometimes referred to as a dual key). If this proved to be a good method, a global whitelist service based on the post-key (simply IP address), sort of like RBLSs but RWLs, could be implemented. I don't know if anyone's pursued such a thing or not. Seems feasible to me though. You are right, but server pools are well known (gmail, yahoo, msn and others) and could be easily discovered and included in a whitelist. Yes, but they change, so you'd need some sort of maintenance procedure to keep them up to date. It's a slow moving target, but far from being fixed. Adding a manual maintenance burden is bad. If it were automated though, that'd be ok. A spammer tends to use only an IP address or few ip addresses, so using a graylist method with single ip addresses could improve security. How would it improve security? Needs explanation. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spamdyke idle timeout problem
Felix Buenemann wrote: Hello, I'd like to ask about your opinion on a good value for idle-timeout-secs – I've started with a value of 60 seconds, which strangely caused TIMEOUTs during mail delivery of large mails (over 10MB) from clinet MTAs with SMTP AUTH (specifically the mail path was: Client MS Exchange (DSL upstream 512 kbps) - Spamdyke SMTP AUTH - qmail - target MTA). I have now raised the idle timeout value to 180 secs, which seems to fix the problem for now, but I wonder what happens with eg. 50 or 100 MB mails. The strange thing is that the idle and not the connection timeout got triggered, because supposedly there is never such a long period of no traffic during sending a large mail. Best Regards, Felix Buenemann What version are you running? There was a bug in most 3.x versions that would timeout on large emails when there was no interruption. Check the documentation's change log to verify which version it was fixed in. There haven't been any reports of this problem with 4.x TTBOMK. Of course, Sam would know for sure. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] DKIM etc.
Eric Shubert wrote: Sam, I see in the TODO file for 4.0 that adding SPF/CSV/Sender ID/DomainKeys/DKIM checking is ranked as a todo-later item. I don't care so much about CSV/SenderID/DomainKeys, but I'd like to see the others implemented sooner than later. In particular, DKIM signatures are reportedly (2/08) being implemented at PayPal and eBay, and I'd expect other (large) financial institutions to be implementing it soon as well. I think it'd be great to have spamdyke rejecting invalid DKIM signatures. This isn't so much simply an anti-spam measure, but a solution to a very real security threat (identity theft). SPF checking is presently available in qmail-toaster, so that's not a high priority for me. However, I think it's more appropriately done by spamdyke than (a patched) qmail, so I'd like to see you do this as well. As far as DomainKeys goes, the qmail-toaster implementation of this, at least on the checking side, is somewhat broken, so it'd be nice to have, but I don't honestly think it's being used, as it's being pretty much replaced with DKIM. My guess is that CSV and SenderID are also not worth the trouble to implement. I hope that others will share their opinions on this as well. I could be wrong (again). ;) Thanks for the great work with spamdyke. FWIW, some surveys regarding mail authentication: http://www.sendmail.org/dkim/surveyFortune1000 http://www.sendmail.org/dkim/surveyUsBanking -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] growing number of tcpservers which eventually reaches the limit of tcpsessioncount
Erald Troja wrote: Hello all, We are using Hsphere control panel automation offered from Parallels with precompiled Qmail binaries. Our entry onto the spamdyke /etc/init.d/qmaild script which is currently running on a CentOS 4.6 is as follows. at the very top we define SPAMDYKE and it's configuration file SPAMDYKE=/usr/local/bin/spamdyke --config-file /etc/spamdyke/spamdyke.conf further down onto the start portion of /etc/init.d/qmaild we issue (all in one line) tcpserver -v $RRDNSKEY -R -c $TCP_SERVERS $IPLIMIT $RELAYCHKARG -u $USER_VPOPMAIL -g $GROUP_VCHKPW 0 smtp $SPAMDYKE $RBL qmail-smtpd vchkpw true cmd5checkpw true 21 | splogger smtpd Our Spamdyke configuration file is as follows. /etc/spamdyke/spamdyke.conf log-level=info graylist-level=always-create-dir graylist-dir=/var/tmp/spamdyke.graylist.d graylist-exception-ip-file=/etc/spamdyke/whitelist.conf graylist-min-secs=1200 graylist-max-secs=4322000 reject-unresolvable-rdns=true reject-empty-rdns=true Our maximum tcpsessioncount is set to 1000. This has been working fine for when our Qmail server was operating without Spamdyke. Recently we've hit the limit of tcpsessioncount twice. I've been monitoring the log files and this happens slowly but surely. I'd like to ask, why, and what can we do to prevent this and make it. Raising tcpsessioncount is an option, yet I believe we will slowly but surely reach the limit as well. Thank you. Try adding: idle-timeout-secs=660 to your configuration file. I'm betting that will fix you up. ;) See http://spamdyke.org/documentation/README.html#TIMEOUTS for details. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] DKIM etc.
Felix Buenemann wrote: Hi, I agree with Arthur and Bgs in that SPF is a smarter thing to check, because it can be done without checking headers and currently has a much wider disribution base. IMHO the only way to properly reject DKIM failed mail is at the end of the DATA command, which is exactly how eg. simscan rejects virii or spam mail. So IMHO DKIM verification is something to do for a queue-handler not a frot end smtp handler, that is geared for high performance. (This is based on the assumtion, that spamdyke deals with 99% of the scam with very little cpu time, thus reducing server load and leaving more in depth checks to those mails that slip through spamdyke's already tight web.) -- Felix Good thinking, Felix. Some things just don't belong in spamdyke as is. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] whitelisting a handful of domains while graylisting others
Erald Troja wrote: Folks, is it possible to simply allow immediate delivery to a handful of domains, while graylisting the rest to the standard defined graylisting policy? Seems some folks would rather just get instant gratification and spam, rather than have a minimal delay with the extra protection graylisting offers. I'm not able to pinpoint it in the docs. I noticed graylist-exception-rdns-dir yet this seems to apply to remote servers, and we're interested into the local domains we receive email for. Thanks. You need spamdyke =4.0 for this. If I understand correctly, use the greylist-level=always in your config, and then be sure that there's no domain directory contained in the greylist-dir directory. See http://spamdyke.org/documentation/README.html#GRAYLISTS -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spamdyke 4.0.4 hanging after denied
Kris Van Hees wrote:
I had a problem where my mail server stopped being able to service connections
because I had as many hanging spamdyke processes as was allowed in my
tcpserver
config (-c option). Unfortunately, the processes were cleaned up by another
admin before I could look at them.
And then, I just ran into the same situation again, where a spamdyke process
is hanging, and has been hanging for 2.5 hours so far. Here is output from
log-level debug in spamdyke (X substituted for domain name):
Oct 5 20:27:16 saffron spamdyke[3978]:
DEBUG(filter_rdns_missing()@filter.c:841): checking for missing rDNS; rdns:
(unknown)
Oct 5 20:27:16 saffron spamdyke[3978]: FILTER_RDNS_MISSING ip: 77.30.98.26
Oct 5 20:27:16 saffron spamdyke[3978]:
DEBUG(filter_ip_whitelist()@filter.c:1120): searching IP whitelist file(s);
ip: 77.30.98.26
Oct 5 20:27:17 saffron spamdyke[3978]:
DEBUG(filter_recipient_relay()@filter.c:2176): checking relaying;
relay-level: 3 recipient: [EMAIL PROTECTED] ip: 77.30.98.26 rdns: (unknown)
local_recipient: true relaying_allowed: false
Oct 5 20:27:17 saffron spamdyke[3978]: DENIED_RDNS_MISSING from: [EMAIL
PROTECTED] to: [EMAIL PROTECTED] origin_ip: 77.30.98.26 origin_rdns:
(unknown) auth: (unknown)
I would have expected the connection to be dropped at this point, and spamdyke
to exit. Looking at lsof -i output for this process, I get:
spamdyke 3978 qmaild0u IPv4 732880026 TCP
saffron.alchar.org:smtp-77.30.98.26:56004 (ESTABLISHED)
spamdyke 3978 qmaild1u IPv4 732880026 TCP
saffron.alchar.org:smtp-77.30.98.26:56004 (ESTABLISHED)
spamdyke 3978 qmaild3u IPv4 732880028 UDP *:41956
So, the connection is still alive. Netstat -an confirms this:
tcp0 0 192.168.0.1:25 77.30.98.26:56004
ESTABLISHED
Looking at strace output, spamdyke is stuck in a select loop, waiting for
something:
Process 3978 attached - interrupt to quit
select(1, [0], NULL, NULL, {1, 58}) = 0 (Timeout)
time(NULL) = 1223261329
select(1, [0], NULL, NULL, {2, 0}) = 0 (Timeout)
time(NULL) = 1223261331
select(1, [0], NULL, NULL, {2, 0}) = 0 (Timeout)
time(NULL) = 1223261333
select(1, [0], NULL, NULL, {2, 0} unfinished ...
Process 3978 detached
Looking at the process using gdb didn't show anything interesting, because the
backtrace is trash (possibly in part due to me stripping the spamdyke
executable). It simply lists the top frame as:
#0 0xb7ec39f8 in select () from /lib/tls/libc.so.6
and the rest if garbage. Smells like possible memory corruption.
Anyone seen something like this? This is with spamdyke 4.0.4.
Kris
Do you have something like
idle-timeout-secs=660
parameter in your configuration? The default is 0, which will not time out.
--
-Eric 'shubes'
___
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Updated Spamdyke Statistics Script
Sergio Minini wrote: Erik, try: # cat /var/log/maillog | ./spamdyke_stats.pl 17661 ALLOWED 14224 DENIED_RBL_MATCH -- Breakdown -- 84.25% zen.spamhaus.org 15.75% bl.spamcop.net --- 12330 DENIED_RDNS_RESOLVE 10299 DENIED_RDNS_MISSING 4296DENIED_GRAYLISTED 651 ERROR 457 DENIED_BLACKLIST_IP 412 DENIED_OTHER 239 TIMEOUT 59 DENIED_SENDER_BLACKLISTED 35 DENIED_TOO_MANY_RECIPIENTS Allowed: 17661 Denied : 42112 Errors : 890 Total : 60663 % Valid: 29.11% % Spam : 69.42% % Error: 1.47% Good luck/ Sergio Thanks, but that doesn't work for me. My spamdyke log messages are going to the qmail smtp log (log-target=0|stderr). Perhaps that's where the problem lies, as the formatting would be slightly different. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Updated Spamdyke Statistics Script
Felix Buenemann wrote: Hi Erik, Am 19.10.2008 3:13 Uhr, Eric Shubert schrieb: Felix Buenemann wrote: Hi Erik, Am 18.10.2008 20:39 Uhr, Eric Shubert schrieb: Sergio Minini wrote: Erik, try: # cat /var/log/maillog | ./spamdyke_stats.pl [...] % Valid: 29.11% % Spam : 69.42% % Error: 1.47% Good luck/ Sergio Thanks, but that doesn't work for me. My spamdyke log messages are going to the qmail smtp log (log-target=0|stderr). Perhaps that's where the problem lies, as the formatting would be slightly different. Yes, the script currently expects syslog syntax. If you can provide a few sample lines from the log and I can modify the script to work with it. -- Felix Thanks, Felix. Here's a sample rejection (each line begins with @4): @400048fa5df51149c60c tcpserver: status: 1/100 @400048fa5df51149d5ac tcpserver: pid 22865 from 209.133.101.250 @400048fa5df51149e164 tcpserver: ok 22865 doris.shubes:192.168.171.11:25 :209.133.101.250::58673 @400048fa5dfc34b1ebec DENIED_SENDER_NO_MX from: [EMAIL PROTECTED] to: [EMAIL PROTECTED] origin_ip: 209.1 33.101.250 origin_rdns: broadcaster.eonline.com auth: (unknown) @400048fa5dfd01593124 tcpserver: end 22865 status 0 @400048fa5dfd015a7d2c tcpserver: status: 0/100 In case you don't know, the first (@4000...) field is a date/time stamp, and can be converted to local time with the tai64nlocal program. I don't know if that'll be necessary or not. Thanks again. I'm eager to see the results. Please test the attached version. You have to pass --nosyslog parameter to the script. Also please check which of the $linematch variants beginning at line 33 is the fastest (run time cat /your/log | spamdyke-stats.pl --nosyslog to find out) and report it back to me. Btw. the new version is about 25% faster than the last one, due to optimizations of the regular expression matching, so updating might be useful to others aswell. -- Felix First $linematch: # time cat /var/log/qmail/smtp/@400048fb90032cd228c4.s | ./spamdyke-stats.pl --nosyslog 372 DENIED_RBL_MATCH -- Breakdown -- --- 366 DENIED_RDNS_RESOLVE 363 DENIED_RDNS_MISSING 242 ALLOWED 213 DENIED_IP_IN_CC_RDNS 104 DENIED_SENDER_NO_MX 45 DENIED_OTHER 1 TIMEOUT Allowed: 242 Denied : 1463 Errors : 1 Total : 1706 % Valid: 14.19% % Spam : 85.76% % Error: 0.06% real0m0.395s user0m0.239s sys 0m0.145s # Second $linematch: # time cat /var/log/qmail/smtp/@400048fb90032cd228c4.s | ./spamdyke-stats.pl --nosyslog 372 DENIED_RBL_MATCH -- Breakdown -- --- 366 DENIED_RDNS_RESOLVE 363 DENIED_RDNS_MISSING 242 ALLOWED 213 DENIED_IP_IN_CC_RDNS 104 DENIED_SENDER_NO_MX 45 DENIED_OTHER 1 TIMEOUT Allowed: 242 Denied : 1463 Errors : 1 Total : 1706 % Valid: 14.19% % Spam : 85.76% % Error: 0.06% real0m0.349s user0m0.231s sys 0m0.109s # Third $linematch: # time cat /var/log/qmail/smtp/@400048fb90032cd228c4.s | ./spamdyke-stats.pl --nosyslog 372 DENIED_RBL_MATCH -- Breakdown -- --- 366 DENIED_RDNS_RESOLVE 363 DENIED_RDNS_MISSING 242 ALLOWED 213 DENIED_IP_IN_CC_RDNS 104 DENIED_SENDER_NO_MX 45 DENIED_OTHER 1 TIMEOUT Allowed: 242 Denied : 1463 Errors : 1 Total : 1706 % Valid: 14.19% % Spam : 85.76% % Error: 0.06% real0m0.331s user0m0.177s sys 0m0.142s # Thanks Felix. BTW, couldn't the script simply test for @ in the first position of any line to determine that it's not a syslog, so the flag wouldn't be necessary? Seems simpler to me. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Updated Spamdyke Statistics Script
Felix Buenemann wrote: Hi Eric, Am 20.10.2008 20:01 Uhr, Eric Shubert schrieb: BTW, couldn't the script simply test for @ in the first position of any line to determine that it's not a syslog, so the flag wouldn't be necessary? Seems simpler to me. This was done to avoid a performance hit of about 20% caused by the more complex regex. However I've now rewritten the detection code, so it's much faster, so the attached version no longer needs (or supports) the --(no)syslog switch. -- -Eric 'shubes' -- Felix Nice. Here's my present result, for all logs: # time cat /var/log/qmail/smtp/*.s | ./spamdyke-stats.pl 34229 DENIED_RDNS_MISSING 26702 DENIED_IP_IN_CC_RDNS 21848 DENIED_RBL_MATCH -- Breakdown -- --- 19514 ALLOWED 14910 DENIED_RDNS_RESOLVE 2684DENIED_SENDER_NO_MX 2123DENIED_OTHER 141 TIMEOUT 3 DENIED_TOO_MANY_RECIPIENTS Allowed: 19514 Denied : 102499 Errors : 141 Total : 122154 % Valid: 15.97% % Spam : 83.91% % Error: 0.12% real0m15.928s user0m4.512s sys 0m3.616s I'm a little confused by the Breakdown in the middle of results. Is the sort not working quite right? Here's what I'd prefer to see as output format: spamdyke-stats.pl v??? Total : 122154 Allowed: 19514 15.97% Timeout: 141 0.12% Denied : 102499 83.91% Denied Breakdown RDNS_MISSING 34229 33.39% IP_IN_CC_RDNS26702 26.05% RBL_MATCH21848 21.32% RDNS_RESOLVE 14910 14.55% SENDER_NO_MX 2684 2.62% OTHER 2123 2.07% TOO_MANY_RECIPIENTS 3 0.00% Or something along those lines. ;) Note the Denied percentages are percents of Denied, not percents of Total. Thanks for your great work on this, Felix. Oh, and one more thing just occurred to me. What about greylist rejections? I'm guessing that these numbers don't take greylisting into account. That would seem to be a bit complicated. Can someone think of a way to account for greylist rejections without complicating things too much? I wonder if Sam couldn't adjust the log messages in such a way that greylisting could be accounted for. Upon further thought, it seems to me that this subject might have been discussed before on the list. Sorry if I'm bringing up a dead horse. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Blacklist Performance question
TazaTek wrote: I have about 1000 IP's in my blacklist_ip file ... and have been adding more every week. At what point does the number of IP's become a performance penalty ? I was trying to reduce the load on the network by taking analyzed RBL matches and place them in the blacklist file but if there becomes a penalty for adding too many IP's then maybe I don't want to do this. Any feed back on when too many is really too many? BTW - I'm on a VPS with 2 GB mem and quad-core CPUS with minimal traffic on the machine, so plenty of horsepower but I'd like to leave it all in reserves for my next Digg/Slashdot opportunity :) Thanks Matt -- I gather that you're trying to reduce the load on the network by essentially using the blacklist_ip file as a sort of RBL facility. Is RBL processing actually creating that much network traffic, or is this just a guess? Do you have a caching nameserver installed on your server? You should, as that will drastically reduce network traffic. How many / which RBLs are you presently using? You shouldn't need more than a few. Also, if you've specified an unresponsive or slow RBL, that can hinder your performance quite a bit. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] simscan, spamassassin and spamdyke
The qmail-toaster (http://qmailtoaster.org) implements these together just fine, so it *is* possible. ;) As Sam said, with a little more info about your configuration I'm expect we can help get you running properly. Sam Clippinger wrote: I'm not familiar enough with simscan to give any good advice here, but spamdyke should work fine with simscan. The FAQ only mentions qmail-scanner because that's what I use and it documents a small bug I found. Perhaps if you could post your run script and more information about your simscan configuration, someone here may be able to help. -- Sam Clippinger Bernd Hoffmann wrote: Hello, I use netqmail 1.06 with tls-smtpauth-patch. I implement clamav and spamassassin with simscan by add :allow,QMAILQUEUE=/var/qmail/bin/simscan to my /etc/tcp.smtp. When I enable spamdyke by adding /usr/local/bin/spamdyke -f /usr/local/etc/spamdyke.conf to my qmail-smtpd-run-script, simscan skip the spamassassin-scan. I don't want to use the qmail-scanner (http://www.spamdyke.org/documentation/FAQ.html#TROUBLE5). That can not be the solution!? How can I encourage simscan to use spamassassin together with spamdyke? Can somebody explain me, what's the problem? Thanks in advance. Best regards, Bernd ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] simscan, spamassassin and spamdyke
Have you compared your simscan configuration to the one used by qmailtoaster.com? Bernd Hoffmann wrote: It doesn't work on my system and I don't understand why. :-( -Ursprüngliche Nachricht- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Hartmut Wernisch Gesendet: Mittwoch, 12. November 2008 14:21 An: spamdyke users Betreff: Re: [spamdyke-users] simscan, spamassassin and spamdyke Here my config files using spamassassin with simscan: [tcp.smtp] :allow,QMAILQUEUE=bin/simscan [simcontrol] xxx.xxx:clam=yes,spam=yes,spam_hits=10 :clam=no,spam=no [smtpd in qmail init file] ulimit -v 16384 sh -c start-stop-daemon --start --quiet --user vpopmail \ --pidfile /var/run/tcpserver_smtpd.pid --make-pidfile \ --exec /usr/bin/tcpserver -- -v -R -H -c 100 \ -u `id -u vpopmail` \ -g `id -g vpopmail` \ -x /etc/tcp.smtp.cdb \ 0 smtp \ /var/qmail/bin/spamdyke --config-file /etc/spamdyke.conf \ /usr/sbin/qmail-smtpd \ /var/vpopmail/bin/vchkpw /bin/true 21 \ | /usr/bin/setuidgid qmaill /usr/bin/multilog \ t n100 s500 /var/log/smtp Sending from my account to my account results in no scanning by spamassassin. Only mails not sending by me with smtp-auth are passed through spamassassin. Hope this helps :) cu, -harti On 12 Nov 08, Bernd Hoffmann wrote: I know the site (http://qmail.jms1.net/simscan) and I patched simscan with simscan-1.4.0-combined.4.patch. I also rebuild tcp.smtp.cdb and simcontrol.cdb after any change of tcp.smtp and simcontrol. Here are the outpot of the logfile without spamdyke in qmailrunfile: simscan: calling clamdscan simscan: cdb looking up version clamav simscan: normal clamdscan return code: 0 simscan: calling spamc simscan: calling /usr/local/bin/spamc spamc -u EMAIL simscan: cdb looking up version spam simscan:[4160]:CLEAN (-101.40/7.00):0.6086s:Testmail:10.10.0.3:EMAILFROM:EMAILTO simscan: done, execing qmail-queue simscan: qmail-queue exited 0 When I add spamdyke to my qmailrunfile, I get the following log entries: simscan: calling clamdscan simscan: cdb looking up version clamav simscan: normal clamdscan return code: 0 simscan:[4213]:RELAYCLIENT:0.0652s:-:10.10.0.3:EMAILFROM:EMAILTO simscan: done, execing qmail-queue simscan: qmail-queue exited 0 As you can see, spamc will not calling by simscan. Can you show me your tcp.smtp file and also your qmail-smtpd-run-script, harti? Thanks in advance. Bernd I have no problem running spamdyke, simscan, clamav, f-prot and spamassassin. Are you sure you rebuild the cdb files after making changes? (tcp.smtp.cdb and simcontrol.cdb) You can enable simscan debugging by adding environmen variable in your /etc/tcp.smtp file: SIMSCAN_DEBUG=2 Maybe you are also interessted in a good site providing some patches for simscan (if you don't know it already:): http://qmail.jms1.net/simscan/ best, -harti On 11 Nov 08, [EMAIL PROTECTED] wrote: thanks for your answers. first of all i would like to say, that simscan works also with spamdyke, but when i add spamdyke to my run-script, simscan skip only spamassassin. and i don't know the reason. My simscan-configuration looks like: ./configure \ --enable-user=simscan \ --enable-clamav=y \ --enable-clamdscan=/usr/local/bin/clamdscan \ --enable-custom-smtp-reject=n \ --enable-per-domain=y \ --enable-attach=y \ --enable-dropmsg=n \ --enable-spam=y \ --enable-spam-passthru=n \ --enable-spamc-user=y \ --enable-spam-hits=7 \ --enable-spamc=/usr/local/bin/spamc \ --enable-qmaildir=/var/qmail \ --enable-workdir=/var/qmail/simscan \ --enable-controldir=/var/qmail/control \ --enable-quarantinedir=/var/qmail/quarantine \ --enable-qmail-queue=/var/qmail/bin/qmail-queue \ --enable-ripmime=/usr/local/bin/ripmime \ --enable-received=y \ --enable-spamassassin-path=/usr/local/bin/spamassassin \ --enable-clamavdb-path=/usr/local/share/clamav \ --enable-sigtool-path=/usr/local/bin/sigtool \ --enable-regex=y My qmail-smtpd-run-script looks like: #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` LOCAL=`head -1 /var/qmail/control/me` if [ -z $QMAILDUID -o -z $NOFILESGID -o -z $MAXSMTPD -o -z $LOCAL ]; then echo QMAILDUID, NOFILESGID, MAXSMTPD, or LOCAL is unset in echo /var/qmail/supervise/qmail-smtpd/run exit 1 fi if [ ! -f /var/qmail/control/rcpthosts ]; then echo No /var/qmail/control/rcpthosts! echo Refusing to start SMTP listener because it'll create an open relay exit 1 fi exec /usr/local/bin/softlimit -m 1000 \ /usr/local/bin/tcpserver -v -R -l $LOCAL -x /etc/tcp.smtp.cdb -c $MAXSMTPD \ -u $QMAILDUID -g $NOFILESGID 0 smtp \ /usr/local/bin/spamdyke -f
[spamdyke-users] Full Log Enhancement
I see in the TODO.txt file, for version N+1 a request to limit full logging by IP or rDNS name. I'd like to see this given a high priority. In addition, I'd like to be able to limit by sender domain. Maybe simply match the right-hand-most portion of the sender's address? (which could be simply domain or entire address) (FWIW) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spam analysis
A possible solution to this problem is to bring your mail server in-house, and/or use an affordable outbound mail service such as DynDNS's Mailhop Outbound. If you don't have a static IP address in-house, DynDNS's CustomDNS service solves that problem affordably. Disclaimer: I'm not associated with DynDNS, but I do use and recommend their services. Joe Canner wrote: Yes, Level 1 protection seems reasonable. We passed level 1 but failed level 2 and 3 because of other clients using our ISP. I've only had one recipient so far block us because of this, but I fear this might be just the beginning. I agree that ISPs should take some responsibility for their clients' spam. I hope our ISP will respond to our complaint. -Original Message- From: spamdyke-users-boun...@spamdyke.org [mailto:spamdyke-users-boun...@spamdyke.org] On Behalf Of Kulkarni Shantanu Sent: Friday, February 27, 2009 14:44 To: spamdyke users Subject: Re: [spamdyke-users] spam analysis yes, but i use their level 1 protection. level 2 3 are indeed aggressive. but i am also of the opinion that isps are partly responsible for their clients using their bandwidth to spam and they should blacklist these customers and take legal action against them. Shantanu -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] My logfile parser (Script)
Sorry to say that I haven't had a chance to check out your script yet, Sebastian. :( Speaking of colored and filtered qmail logfiles though, there's a nice 'qmlog' script at qtp.qmailtoaster.com (part of the qmailtoaster-plus package). It allows easy viewing and searching of qmail (et al) logs. I'm wondering if your 'coloring and filtering' might be a nice enhancement to that script. Care to have a look into it? Sebastian Grewe wrote: I totally forgot about that - but I am not using the script to block them forever, just to monitor qmail when a large amount of connections is coming in (which happens ever so often). Even so I did turn off the blocking feature since qmail handles it just fine and connections clear up after a while. I was just concerned that legitimate e-mail wouldn't be coming through - but since they try to resend if no connection could be established that's not a concern anymore. So yeah, I use it to see what's being blocked and for what reason - even added whitelist matches now. It's basically just colored and filtered output of your qmail logfiles now :D Cheers, Sebastian Otto Berger wrote: you could also use fail2ban for that. You just have to specify a custom rule (filter) for the spamdyke-log output. Then the sender ip will be released after a specified timeframe and not blocked forever ;). (IMHO it is still not a very good idea to block by firewall) Otto Sebastian Grewe schrieb: Hey Guys, I have been working on a simple bash script that will read from it's standard input and presents some statistics from the logfile in realtime (when used with tail -f .. ). After a few days that we have been attacked by spambots I got curious how to avoid these things in the future. The script we use is able to count the denied connections per IP and, if desired, adds this IP to the Firewall to reject incoming connections (brutal, I know). As the firewalling is optional you might still be interested in it to run just to see what's going on. It's written for BASH 3.0.15 but with a little change in the pattern matcher it runs on higher versions too. To start it in live mode run it like this: tail -f /var/log/qmail/smtp/current | qmail_parser.sh and if you just want to scan some files and see what happened to this: cat /var/log/qmail/smtp/* | qmail_parser.sh Since it's BASH it's not very good when it comes to performance but does the trick well when used with tail. Also it's not catching everything (yet) since I was looking for only some very specific lines in the logfile. Anyhow, try it out and tell me what you think - attached the current script to this mail. Cheers, Sebastian ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spam analysis
I've found that using DynDNS services along with a regular dynamic address is more affordable than going the static IP route (no pun intended). Such a service level is adequate for most SMBs, but not appropriate for mission critial environments. Joe Canner wrote: Sorry, I ignored the first part of your post. Our mail server is already in-house with a static IP. However, the outbound mail service idea may be a useful way to approach this problem. If it gets worse and the ISP doesn't do anything about it, that may be worth investigating. -Original Message- From: spamdyke-users-boun...@spamdyke.org [mailto:spamdyke-users-boun...@spamdyke.org] On Behalf Of Eric Shubert Sent: Friday, February 27, 2009 15:40 To: spamdyke-users@spamdyke.org Subject: Re: [spamdyke-users] spam analysis A possible solution to this problem is to bring your mail server in-house, and/or use an affordable outbound mail service such as DynDNS's Mailhop Outbound. If you don't have a static IP address in-house, DynDNS's CustomDNS service solves that problem affordably. Disclaimer: I'm not associated with DynDNS, but I do use and recommend their services. Joe Canner wrote: Yes, Level 1 protection seems reasonable. We passed level 1 but failed level 2 and 3 because of other clients using our ISP. I've only had one recipient so far block us because of this, but I fear this might be just the beginning. I agree that ISPs should take some responsibility for their clients' spam. I hope our ISP will respond to our complaint. -Original Message- From: spamdyke-users-boun...@spamdyke.org [mailto:spamdyke-users-boun...@spamdyke.org] On Behalf Of Kulkarni Shantanu Sent: Friday, February 27, 2009 14:44 To: spamdyke users Subject: Re: [spamdyke-users] spam analysis yes, but i use their level 1 protection. level 2 3 are indeed aggressive. but i am also of the opinion that isps are partly responsible for their clients using their bandwidth to spam and they should blacklist these customers and take legal action against them. Shantanu -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] My logfile parser (Script)
Thanks. I'm sure you'll keep us posted! :) Sebastian Grewe wrote: After checking out the code in that script I think it might be easier for me to just start on my script and extend it's functionality to look for all lines in those logfiles instead of just spamdyke. I will see what I can do. Cheers, Sebastian Eric Shubert wrote: Sorry to say that I haven't had a chance to check out your script yet, Sebastian. :( Speaking of colored and filtered qmail logfiles though, there's a nice 'qmlog' script at qtp.qmailtoaster.com (part of the qmailtoaster-plus package). It allows easy viewing and searching of qmail (et al) logs. I'm wondering if your 'coloring and filtering' might be a nice enhancement to that script. Care to have a look into it? Sebastian Grewe wrote: I totally forgot about that - but I am not using the script to block them forever, just to monitor qmail when a large amount of connections is coming in (which happens ever so often). Even so I did turn off the blocking feature since qmail handles it just fine and connections clear up after a while. I was just concerned that legitimate e-mail wouldn't be coming through - but since they try to resend if no connection could be established that's not a concern anymore. So yeah, I use it to see what's being blocked and for what reason - even added whitelist matches now. It's basically just colored and filtered output of your qmail logfiles now :D Cheers, Sebastian Otto Berger wrote: you could also use fail2ban for that. You just have to specify a custom rule (filter) for the spamdyke-log output. Then the sender ip will be released after a specified timeframe and not blocked forever ;). (IMHO it is still not a very good idea to block by firewall) Otto Sebastian Grewe schrieb: Hey Guys, I have been working on a simple bash script that will read from it's standard input and presents some statistics from the logfile in realtime (when used with tail -f .. ). After a few days that we have been attacked by spambots I got curious how to avoid these things in the future. The script we use is able to count the denied connections per IP and, if desired, adds this IP to the Firewall to reject incoming connections (brutal, I know). As the firewalling is optional you might still be interested in it to run just to see what's going on. It's written for BASH 3.0.15 but with a little change in the pattern matcher it runs on higher versions too. To start it in live mode run it like this: tail -f /var/log/qmail/smtp/current | qmail_parser.sh and if you just want to scan some files and see what happened to this: cat /var/log/qmail/smtp/* | qmail_parser.sh Since it's BASH it's not very good when it comes to performance but does the trick well when used with tail. Also it's not catching everything (yet) since I was looking for only some very specific lines in the logfile. Anyhow, try it out and tell me what you think - attached the current script to this mail. Cheers, Sebastian ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] E-mail quarantine
Joe Canner wrote: Dear Spamdyke community, Is it possible to use Spamdyke to quarantine e-mail, e.g., identify all mail from a certain sender (or other criteria) and send it to a special mailbox rather than deliver it to the intended recipient? If not, what other tools are available to do this? Thanks for your help. Joe Canner I'm not aware of any spamdyke capability to do this. The closest thing I know of is http://www.inter7.com/?page=qmailtap. That'll make a copy, but won't prohibit delivery to the intended recipient. Perhaps you could patch it to do so. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spamdyke Timeouts for no reason
Sebastian Grewe wrote: .. And here the output from your command below: sa-learn -u vpopmail --force-expire bayes: synced databases from journal in 0 seconds: 1512 unique entries (1901 total entries) So if my program syncs it in 0 seconds I would assume that spamassassin will take roughly the same time when booting up. Looks like the mail itself is the cause for the hickup? Syncing the journal (which is always pretty quick) happens automatically with bayes expiration, but is a different process. You'd need to time the command to get an accurate number. I expect that force-expire will run quicker when it has been run recently. When bayes_autoexpire is on, it is only run on occasional messages (and infrequently), not every one. Sebastian PS: Did you have a look at the colored version of qmlog already? I got a patch flying around somewhere still ... Not yet. Thanks for the reminder. Eric Shubert wrote: FWIW, if you're using spamassassin (along with simscan), you might want to be sure that you have bayes_auto_expire 0 in your local.cf file. This function can take several minutes when it kicks in, and occurs while the smtp session is still active (possibly resulting in duplicate messages). You can simply set up a cron job to expire bayes database entries instead, e.g. (w/ vpopmail as mail user): sa-learn -u vpopmail --force-expire chown vpopmail:vchkpw /home/vpopmail/.spamassassin/bayes_toks Sebastian Grewe wrote: .. Sorry for this but I found the solution here: http://www.qmailwiki.org/index.php/Simscan/README#How_to_Disable.2FEnable_simscan_for_smtp_connections_by_IP_ranges I removed simscan for the know IP's and now mail is coming through. Let's hope it's just that one. Thanks anyway and maybe this information could be useful to someone else out there :) Cheers, Sebastian Sebastian Grewe wrote: Hey again, I was a bit quick on the draw there. After some research on our end I noticed that the spam filter needs quite some time to actually scan that mail (60s) which results in the provider to time us out. So my question is: How can I skip simscan from processing messages from that one provider, given I have their IPs? I would do that until the message comes through and then disable the whitelist after that. Cheers, Sebastian Sebastian Grewe wrote: Hey guys, Here an issue we had a long time ago already: A mail provider connects to our system to deliver a mail. Spamdyke accepts the message and sends it over to Simscan. Spamdyke logs a TIMEOUT error in our Log, right after that Simscan completes the scan. It looks like it hangs somewhere so Spamdyke, by mistake, sends a Timeout to the ISP. That one tries to send the same message again which of course fails again. Just wondering, if anyone else had this issue? If not I can post more details about it. Cheers, Sebastian ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] dumb question - redirect version info to file
Did you have a look at qtp-install-spamdyke?
# determine which version is already installed
#
a4_check_installed_version(){
sdver=$(spamdyke -v 21)
rc=$?
if [ $rc == 0 ]; then
sdverstring=$(echo $sdver | sed -e 's/^spamdyke //')
instver=${sdverstring%%.*}
else
instver=0
fi
}
;)
dnk wrote:
Bingo, bango, Sugar in the gas tank.
Works like a charm.
d
On 23-Apr-09, at 2:13 PM, Sebastian Grewe wrote:
Make that
spamdyke -v 2 version.txt
Cheers,
Sebastian
dnk wrote:
Hi there,
I am writing a home brew report on my qmail machines. I would like to
include the spamdyke version. So I tried:
spamdyke -v report.txt
No matter what I do, I can not get it to redirect the output to a
file.
D
___
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
___
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
--
-Eric 'shubes'
___
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Is there a way to populate thegraylistdatabase WITHOUT effectively doing graylisting
David Sánchez Martín wrote: David, That sounds like a neat idea, but I don't think it'd work. If you simply allow the session to complete and create a greylist entry for everything, you will have effectively whitelisted every incoming message, including the bad ones. Greylisting works because some spammers don't retry when a session fails. If everything passes, you've no way of knowing which ones would or would not have retried. The greylist database would be useless. Let me think about it. If greylisting is enabled as usual: When a foreign user sends a message to a local user is greylisted, then: 1.- It's created an entry in the greylisting database. 2.- It's blocked and each retry is blocked also at least for graylist-min-secs seconds. 3.- No further tests are passed. Session is closed. When graylist-min-secs time passes: 1.- The message passes greylist filter and touches the file. 2.- The message is tested against other filters. Ok, What i'm trying to accomplish: When a user foreign a message to a local then: 1.- The message passes greylist filter and touches the file. 2.- The message is tested against other filters. That will populate the database, that is what i want before putting graylist at work. Sorry, perhaps I'm missing something. Best regards. That will populate the database for all email. Including spammers. Any spammers who send messages during the period in which the database is being populated will get a free pass, even after greylisting is activated. Perhaps you can live with that. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Is there a way to populate thegraylistdatabase WITHOUT effectively doing graylisting
David Sánchez Martín wrote: That will populate the database for all email. Including spammers. Any spammers who send messages during the period in which the database is being populated will get a free pass, even after greylisting is activated. Perhaps you can live with that. That will populate the database with all the addresses who send email to my users. Including spam. Just like the graylisting do, no more no less. The entry will survive _as_long_as_it_will_with_graylisting_fully_enabled_, NO MORE and no less. It will NOT whitelist the address. Right, but some spammers will be passing the greylist. After graylisting been enabled, It won't block addresses already on the database AND that its time is lesser than graylist-max-secs. No more and no less. About graylist-max-secs (from the doc): NOTE: A graylist entry's expiration date is reset each time a message passes the filter. If the maximum age is 2 weeks and the sender sends a message every day, their entry will never expire because it is continually reset. Given that your primary objective seems to be to eliminate any delays from existing emailers, I suppose this would work for you. Spammers who hit sporadically will eventually expire. I just intend to point out that persistent spammers who send more often than graylist-max-secs will continue to pass. Again, this might be livable. I've no idea how persistent spam generally is. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Is there a way to populate thegraylistdatabase WITHOUT effectively doing graylisting
Thanks, David. The light just came on. (duh) :) David Sánchez Martín wrote: Given that your primary objective seems to be to eliminate any delays from existing emailers, I suppose this would work for you. Spammers who hit sporadically will eventually expire. I just intend to point out that persistent spammers who send more often than graylist-max-secs will continue to pass. Again, this might be livable. I've no idea how persistent spam generally is. That's correct, and it's true for the whole graylisting process. There's no difference, to this extend, of enabling it in full at the very beginning or not. Persistent spammers will hit, in any case, but that wasn't what I was trying to solve (as you said, this is something I should consider if it's acceptable or not, but this is another matter, graylisting is what it is, you can take it or leave it as is). Best regards :-) --- David Sanchez Martin Administrador de Sistemas dsanc...@e2000.es GPG Key ID: 0x37E7AC1F E2000 Nuevas Tecnologías Tel : +34 902 830500 ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] graylist bounces to sender
Greg Cirelle Enterprises wrote: Is there a common reason why the sender of an email would receive a graylist bounce message? spamdyke conf graylist-level=always graylist-min-secs=290 graylist-max-secs=61600 They haven't authenticated successfully? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] graylist bounces to sender
Greg Cirelle Enterprises wrote: Eric Shubert wrote: Greg Cirelle Enterprises wrote: Is there a common reason why the sender of an email would receive a graylist bounce message? spamdyke conf graylist-level=always graylist-min-secs=290 graylist-max-secs=61600 They haven't authenticated successfully? these are external users sending to our users, so there is no authentication at our end Oh, I thought you meant submitters. That wouldn't be a bounce though. Can you get an example of the bounce? What does smtp log show for the corresponding message? Is the sending server re-trying? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail writes with wrong user to the maildir
Stefan Pausch wrote: Hello, i know this is not a spamdyke issue, but since here are very smart heads i thought i give it a try and I hope you don’t mind. I posted already on 3 forums and contacted my provider and plesk support … with no solution at all. My system configuration: - Plesk 9.2.1 with QMail und Spamdyke (+Mysql) - Debian Sarge 64bit ( 2.6.18-6-amd84 ) - xinetd My .qmail configuration: | true | /usr/bin/deliverquota ./Maildir Maildir is: /var/qmail/mailnames/DOMAIN.tld/USER/Maildir/new My problem is that “deliverquota” writes new emails with the wrong username (root:popuser instead of popuser:popuser) into the maildirs (which causes issues). Does anybody here know where I can configure which user:group is used? … this drives me nuts for a few weeks (currently a 1min cronjob is running to correct this issue *sigh) Thanks a lot for any help. --Stefan I'm not familiar with Plesk, but I believe that if you set the sticky bit on the email folder (/var/qmail/mailnames/DOMAIN.tld/USER/Maildir/new), then the individual emails will be created with the owner of that folder instead of the owner of the process that runs deliverquota. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spamdyke Timeout
Ronnie Tartar wrote: I have spamdyke in front of Qmail Toaster and this morning all incoming emails were being timed out. I have had to temprarily remove Spamdyke from the server, ugh, already getting more spam. From the maillog: Jul 13 06:59:28 mail spamdyke[21362]: TIMEOUT from: (unknown) to: (unknown) origin_ip: 72.29.91.100 origin_rdns: prod12.designatedystems.com auth: (unknown) reason: TIMEOUT The above server is whitelisted in my config file. The reverse dns works fine, after removing spamdyke out of the process all email comes in. spamdyke config-test returns nothing. OS CentOS 5.2 64bit. Everything was working fine for 4+ months @ least. Thanks in advance. We'll take this up on the qmail-toaster list. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] smtp authentication
Port 25 needs to accept email for local domains without authentication so that incoming mail can be delivered. email coming into port 25 for remote domains (relay) should be rejected under normal circumstances, otherwise your server would be an open relay. If you want all users to authenticate even when they're sending to local domains, then you need to configure the submission port 587 for such use. That is what port 587 is for. Which flavor of qmail are you running? (QMT, QMR, LWQ) ramalingam m wrote: Dear all, I have tried with qmail setup it was nice but for smtp authentication i found spamdyke will be nice tool so i configured spamdyke installation very easy the authentication i am unable to configure it properly i configured authentication the authentication is working but the mail is also going even if the user is not authenticated i want the users to authenticate and then only they have to send mail. I tried this option filter-level=require-auth problem is mail from outside is not reaching my domain saying authentication error. host test.mail.com http://test.mail.com[192.168.8.112] said: 554 Refused. Authentication is required to send mail. (in reply to RCPT TO command) -- my /etc/spamdyke.conf file is log-level=info local-domains-file=/var/qmail/control/rcpthosts max-recipients=5 #rejection-text-smtp-auth-required=TEXT access-file=/etc/tcp.smtp smtp-auth-level=always smtp-auth-command=/home/vpopmail/bin/vchkpw /bin/true filter-level=require-auth -- my /var/qmail/supervise/qmail-smtpd/run file is #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` LOCAL=`head -1 /var/qmail/control/me` if [ -z $QMAILDUID -o -z $NOFILESGID -o -z $MAXSMTPD -o -z $LOCAL ]; then echo QMAILDUID, NOFILESGID, MAXSMTPD, or LOCAL is unset in echo /var/qmail/supervise/qmail-smtpd/run exit 1 fi if [ ! -f /var/qmail/control/rcpthosts ]; then echo No /var/qmail/control/rcpthosts! echo Refusing to start SMTP listener because it'll create an open relay exit 1 fi exec /usr/local/bin/softlimit -m 3000 \ /usr/local/bin/tcpserver -v -R -l $LOCAL -x /etc/tcp.smtp.cdb -c $MAXSMTPD \ -u $QMAILDUID -g $NOFILESGID 0 smtp \ /usr/local/bin/spamdyke -f /etc/spamdyke.conf \ /var/qmail/bin/qmail-smtpd test.mail.com http://test.mail.com \ /home/vpopmail/bin/vchkpw /usr/bin/true 21 Please help me. Regards M. Ramalingam ~ -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Exchange Replacement [was: where is Sam?]
dnk wrote: On 2009-07-15, at 7:56 PM, sebasti...@jammicron.com wrote: Glad you are still around, would be a shame to see this excellent piece of software go down in inactivity! It is one of the main reasons I have not yet jumped to another mail server from qmail. My company wants an exchange replacement. d I'm curious to know why your company wants Exchange. Some (dis)functionality that you don't presently have? What are the reasons? If you end up having to install Exchange as a mail store, you're still going to need some anti-spam measure. A front-end mail server (QMT) with spamdyke suits this purpose nicely. So I'm guessing that you'll simply end up adding an exchange server to what you have, and migrating the user accounts to exchange. You could probably even run them both on the same hardware with VMware if you like (depending on your volume). -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Exchange Replacement [was: where is Sam?]
dnk wrote: On 2009-07-16, at 10:11 AM, Eric Shubert wrote: dnk wrote: On 2009-07-15, at 7:56 PM, sebasti...@jammicron.com wrote: Glad you are still around, would be a shame to see this excellent piece of software go down in inactivity! It is one of the main reasons I have not yet jumped to another mail server from qmail. My company wants an exchange replacement. d I'm curious to know why your company wants Exchange. Some (dis)functionality that you don't presently have? What are the reasons? If you end up having to install Exchange as a mail store, you're still going to need some anti-spam measure. A front-end mail server (QMT) with spamdyke suits this purpose nicely. So I'm guessing that you'll simply end up adding an exchange server to what you have, and migrating the user accounts to exchange. You could probably even run them both on the same hardware with VMware if you like (depending on your volume). -- -Eric 'shubes' Well they do not want an exchange box per se, they want exchange features (IE global address in the client, shared calendars, etc - In fact that is all they want. I don't even need the rest.). What I meant by Exchange Replacement was one of the linux packages that were an Exchange Replacement - such as Scalix, Zimbra, etc. Have a look at eGroupWare. I'm presently looking into integrating it with QMT. I myself just haven't had much time or success in finding a good calendar server for CentOS (meaning easy to install or update). Nor have I tackled LDAP yet for a GAB (global address book). Just more time restrictions on my part. My intent was to always document it and throw it on the wiki if I ever got one working (either on a toaster box, or separate). Then it could help others like me who want to keep running the toaster but have pressure from the higher level to add more exchange like features. D You're not the only one. I'm working on it. Stay tuned... -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] noob question
Les Fenison wrote:
I just installed spamdyke on my server running Plesk 9.2.2
I can not tell if it is actually running as it is logging nothing. I
blacklisted my own IP for a test and it didn't stop me from sending, of
course I was authenticated so maybe that was normal.
Right, authenticated sessions are not blocked in any way.
I am starting out conservative with nothing in my config file except
log-level=info
log-target=syslog
ip-blacklist-entry={my IP here}
My /etc/xinit.d/smtp_psa file looks like this...
service smtp
{
socket_type = stream
protocol= tcp
wait= no
disable = no
user= root
instances = UNLIMITED
env = SMTPAUTH=1
server = /var/qmail/bin/tcp-env
server_args = -Rt0 /usr/local/bin/spamdyke
/var/qmail/bin/relaylock /var/qmail/bin/qmail-smtpd
/var/qmail/bin/smtp_auth /var/qmail/bin/true /var/qmail/bin/cmd5checkpw
/var/qmail/bin/true
}
Any idea why spamdyke doesn't seem to be running? How can I tell if it is?
I don't know plesk (I run qmail-toaster), but spamdyke on plesk would
log in /usr/local/psa/var/log/maillog. See
http://www.spamdyke.org/documentation/README.html#LOG
I would expect to see an ALLOWED message from spamdyke.
--
-Eric 'shubes'
___
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] newbie question - please bear with me - some Spam getting through
Christoph Kuhle (Expat Email Ltd) wrote: Separately, I do notice a small but sufficiently significant number of genuine emails which get rejected with no reverse DNS. Should we be happy to put email addresses on the white list, or is that dangerous with Spammers being able to get through if they purport to be that address? Up to now, we have just passed on the maillog entry on so that they can check it out with their own hosting company. This is what I do, whitelist and notify the sending server's admin. It'd be nice if there was a spamdyke tool that would allow one to easily re-check an IP address to see if their server has subsequently been fixed, as an aid in keeping the whitelist clean. Sort of a if a certain IP address were to send an email to my server, would spamdyke reject it? tool. What do you think, Sam? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] newbie question - please bear with me - some Spam getting through
I use @domain.com in whitelist_senders file and it works. I think it needs to have the @ sign. Christoph Kuhle (Expat Email Ltd) wrote: Thank you Eric, Interesting one. I put the whole domain in /var/qmail/spamdyke/whitelist_senders by simply putting revivevending.com in that file. I seem to remember reading that this is possible. The restarted Apache /etc/init.d/httpd restart but it was still being rejected. Then I put in the full email address, and it worked. Is it possible to put a whole domain in whitelist_senders? Kind regards, Christoph -Original Message- From: spamdyke-users-boun...@spamdyke.org [mailto:spamdyke-users-boun...@spamdyke.org] On Behalf Of Eric Shubert Sent: 26 August 2009 15:13 To: spamdyke-users@spamdyke.org Subject: Re: [spamdyke-users] newbie question - please bear with me - some Spam getting through Christoph Kuhle (Expat Email Ltd) wrote: Separately, I do notice a small but sufficiently significant number of genuine emails which get rejected with no reverse DNS. Should we be happy to put email addresses on the white list, or is that dangerous with Spammers being able to get through if they purport to be that address? Up to now, we have just passed on the maillog entry on so that they can check it out with their own hosting company. This is what I do, whitelist and notify the sending server's admin. It'd be nice if there was a spamdyke tool that would allow one to easily re-check an IP address to see if their server has subsequently been fixed, as an aid in keeping the whitelist clean. Sort of a if a certain IP address were to send an email to my server, would spamdyke reject it? tool. What do you think, Sam? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] newbie question - please bear with me - some Spam getting through
That's what I was looking for (I think). I should be able to script something together with that. I'll post it here if/when I get it done. Not really a high priority at the moment. ;) Sam Clippinger wrote: That's not a bad idea, I'll add that to the ever-growing list. :) With the current version (assuming you're comfortable at the command line), you can set the TCPREMOTEIP environment variable to the remote IP and run spamdyke manually to see what it says. Something like this: $ export TCPREMOTEIP=11.22.33.44 $ spamdyke -f /etc/spamdyke.conf /var/qmail/bin/qmail-smtpd /bin/true -- Sam Clippinger Eric Shubert wrote: Christoph Kuhle (Expat Email Ltd) wrote: Separately, I do notice a small but sufficiently significant number of genuine emails which get rejected with no reverse DNS. Should we be happy to put email addresses on the white list, or is that dangerous with Spammers being able to get through if they purport to be that address? Up to now, we have just passed on the maillog entry on so that they can check it out with their own hosting company. This is what I do, whitelist and notify the sending server's admin. It'd be nice if there was a spamdyke tool that would allow one to easily re-check an IP address to see if their server has subsequently been fixed, as an aid in keeping the whitelist clean. Sort of a if a certain IP address were to send an email to my server, would spamdyke reject it? tool. What do you think, Sam? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail + spamdyke + chkuser
Youri V. Kravatsky wrote: Hello, people! :) Probably I've missed something, but there is any means to integrate qmail, spamdyke and chkuser (mine was with quotacheck)? I don't want to reinstall/replace qmail (well, 215 domains are hosted at this server). Right now all messages that have passed spamdyke are trying to be delivered, even to non-existant users/domains. It's very annoying (queue is loaded with junk for 1500-1600 messages permanently). I've searched in mailing list, but didn't found anything :( I think that checking for the existence of users is a future feature of spamdyke, and is presently handled by the delivery end of things, after the message has been accepted from the sending server. Your undeliverable messages should expire from the local delivery queue after they've reached the age specified in your queuelifetime file. You might want to shorten the time (# of seconds) specified there. If you're using vpopmail, qmailadmin allows you to set your catchall action on the email accounts page. Without qmailadmin, you'll need to edit the .qmail-default file for each domain manually. You can either delete or bounce as a catchall action. Deleting is preferred, as bouncing contribute to backscatter spam. HTH -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] check for mail from email address
Rajesh M wrote: hello i noted that as long as i smtp authenticate qmail does not check to ensure that the mailfrom email id (domain name part) exists in the rcpthosts file or not. i need to check to ensure that the domain part of the mailfrom email id is a domain hosted on my server. example if domain1.com is hosted on on my server and therefore present in rcpthosts file. and u...@domain1.com is sending an email after smtp authentication. then while accepting this email i would need qmail smtp to check to ensure that domain1.com is exists in the rcpthosts file. now the question is this possible by modifying the spamdyke file, if possible, any tips on this would be helpful ? thanks rajesh This is typically done with the eMPF facility, as Jake pointed out on the QMT list. Please wait a bit before cross posting, rajesh. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Order of processing
Ronnie Tartar wrote: Is there an order to how the different configuration parameters are executed. For instance, the white/black lists are processed, then dns tests then the last is the rbl's? Yes. See http://spamdyke.org/documentation/FAQ.html#FEATURE1 Does it matter what order they are in the config file? Not most of the time. Alphabetically works nicely so you can find them easily. Order is significant only when a given parameter is listed more than once. (I'm going out on a limb here - Sam will correct me when I'm wrong. ;) ) Thanks Welcome. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail + spamdyke + chkuser
Youri V. Kravatsky wrote: Hello Sam, Wednesday, September 2, 2009, 12:05:59 AM, you wrote: chkuser is just another filter that intercepts the data before qmail sees it, so I don't see any reason it won't work with spamdyke. IIRC, QmailToaster uses both chkuser and spamdyke. When chkuser rejects a recipient, spamdyke should log it with DENIED_OTHER. How should I activate chkuser with spamdyke? Add chkuser before/after spamdyke in the shell script that starts qmail? Or spamdyke has means to call to additional filters from its config? chkuser is implemented via a patch to qmail. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spam Stats
Sergio Minini (NETKEY) wrote: Mirko Buffoni escribió: div class=moz-text-flowed style=font-family: -moz-fixedGoods average between 500 and 2000 daily. Figures are however pretty standard. Spamdyke filters out about 60k attempts daily. Here are yesterday stats: Good : 1025 = 0.68 % Unsure :183 = 0.12 % Virus : 62 = 0.04 % BAD Sender: 5114 = 3.40 % BAD Rcpt :212 = 0.14 % Pure SPAM : 45997 = 30.56 % SPAMMER : 97940 = 65.06 % | \.BLACKLISTED_KEYWORD : 29608 = 30.23 % \..DENIED_EARLYTALKER : 3 = 0.00 % \...DENIED_IP_IN_RDNS : 30447 = 31.09 % \DENIED_RBL_MATCH : 23268 = 23.76 % \.DENIED_SENDER_NO_MX : 13070 = 13.34 % \..DENIED_TOO_MANY_RECIPIENTS : 1 = 0.00 % \DENIED_UNQUALIFIED_RECIPIENT : 1 = 0.00 % \.TIMEOUT : 1542 = 1.57 % -- Total : 150533 = 100.00 % Mirko, nice layout of stats. Could you please share the script you are using to get them? Thanks! -Sergio Ditto! Somebody did a nice job! (I wonder if this is this based on the spamdyke-stats.pl script that Felix Buenemann did last October) Pleeeze Mirko? I'd like to include in with the qmailtoaster-plus scripts. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spam Stats
Mirko,
That answers the 'pretty formatting' part, but the meat of the sandwich
is collecting the stats. I'm afraid that Virus stats are collected
through clamav, bad_sender/rcpt are chkuser GREPs, and so on leaves us
hanging. :(
The data collection code is what I'm most interested in. Are the stats
gathered continually and stored, or are they gathered dynamically on
demand? This is the code I'm most interested in. The $spamdyke part is
particularly mysterious. If it's a bit disjointed that's ok. I'm sure
that we can work with it.
Thanks again.
Mirko Buffoni wrote:
Sergio, Eric,
It's nothing really worth worldwide attention. It's a simple php
script that collects data from various sources and aggregates them.
Here is the relevant part:
$res = sprintf( Antispam Statistics for: .date('d/m/Y',
time()-86400).
\n\n.
Good : % 6d = %6.2f %%\n.
Unsure : % 6d = %6.2f %%\n.
Virus : % 6d = %6.2f %%\n.
BAD Sender: % 6d = %6.2f %%\n.
BAD Rcpt : % 6d = %6.2f %%\n.
Pure SPAM : % 6d = %6.2f %%\n.
SPAMMER : % 6d = %6.2f %%\n%s.
--\n.
Total : % 6d = 100.00 %%\n\n,
$pure_good, 100.0 * $pure_good / $total_mails,
$unsure, 100.0 * $unsure / $total_mails,
$virus, 100.0 * $virus / $total_mails,
$pure_spam, 100.0 * $pure_spam / $total_mails,
$bad_sender, 100.0 * $bad_sender / $total_mails,
$bad_rcpt, 100.0 * $bad_rcpt / $total_mails,
$intrusion, 100.0 * $intrusion / $total_mails,
$spamdyke,
$total_mails );
It's not based on any other statistics script, as it need to serve only
my own purposes. Virus stats are collected through clamav, bad_sender/rcpt
are chkuser GREPs, and so on.
Mirko
At 16:10 02/09/2009 -0700, you wrote:
Sergio Minini (NETKEY) wrote:
Mirko Buffoni escribió:
div class=moz-text-flowed style=font-family: -moz-fixedGoods
average between 500 and 2000 daily. Figures are however
pretty standard. Spamdyke filters out about 60k attempts daily.
Here are yesterday stats:
Good : 1025 = 0.68 %
Unsure :183 = 0.12 %
Virus : 62 = 0.04 %
BAD Sender: 5114 = 3.40 %
BAD Rcpt :212 = 0.14 %
Pure SPAM : 45997 = 30.56 %
SPAMMER : 97940 = 65.06 %
|
\.BLACKLISTED_KEYWORD : 29608 = 30.23 %
\..DENIED_EARLYTALKER : 3 = 0.00 %
\...DENIED_IP_IN_RDNS : 30447 = 31.09 %
\DENIED_RBL_MATCH : 23268 = 23.76 %
\.DENIED_SENDER_NO_MX : 13070 = 13.34 %
\..DENIED_TOO_MANY_RECIPIENTS : 1 = 0.00 %
\DENIED_UNQUALIFIED_RECIPIENT : 1 = 0.00 %
\.TIMEOUT : 1542 = 1.57 %
--
Total : 150533 = 100.00 %
Mirko, nice layout of stats.
Could you please share the script you are using to get them?
Thanks!
-Sergio
Ditto! Somebody did a nice job!
(I wonder if this is this based on the spamdyke-stats.pl script that
Felix Buenemann did last October)
Pleeeze Mirko? I'd like to include in with the qmailtoaster-plus scripts.
--
-Eric 'shubes'
___
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
--
-Eric 'shubes'
___
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spam Stats
Mirko Buffoni wrote: Hi Eric, At 06:50 03/09/2009 -0700, you wrote: Mirko, That answers the 'pretty formatting' part, but the meat of the sandwich is collecting the stats. I'm afraid that Virus stats are collected through clamav, bad_sender/rcpt are chkuser GREPs, and so on leaves us hanging. :( You can collect data in a various amount of ways. For continuous collection I suggest to use collectd package, altough for spam/mail statistics I'm afraid you'll have to write your own plugins. To count the entries in a daily rotated log file a simple grep VIRUS FOUND clamav/current.1 | wc -l is enough. The same applies to other patterns in the log file. I'm very familiar with this sort of thing. The data collection code is what I'm most interested in. Are the stats gathered continually and stored, or are they gathered dynamically on Since they are a daily statistic, they are collected after logfile rotation and stored/processed. Can you share the code that does this collecting and storing?? Mirko -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spam Stats
I don't have any FILTER_RBL messages. I'm using log-level=2. What log level are you using? I think that it's appropriate to count each recipient as a separate email. If the message came from a qmail server, it would be that way anyhow. And after all, that's how many messages end up being delivered. Sebastian Grewe wrote: Hey list, I just looked at those stats and compared the output to what I am having on our boxes and I started wondering: When I check the log files, Spamdyke logs the following FILTER_RBL_MATCH : When listed in the RDNS DENIED_RBL_MATCH : For each recipient address in the mail So basically it will result in 1 FILTER match but 1 DENIED match for each mail address. Doesn't that mean that using the DENIED match will not result in the actual denied mails but rather in a much higher number? I am currently looking for both FILTER_ and DENIED_ flags and sum those up to find out how many mails I rejected - but I am guessing here that looking for FILTER_ alone would make more sense. Here my output, wrote the script today - Mirkos' output inspired me :) It's tailored to work for our environment though. Total : 1571(100.%) Legitimate : 123 (7.8200%) | |-FILTER_WHITELIST : 61 (49.5900%) | |-_RECIPIENT_WHITELIST : 61 (100.%) Rejected : 1448 (92.1700%) | |-FILTER : 539 (37.2200%) || ||- _RDNS_MISSING : 192 (35.6200%) ||- _OTHER: 12 (2.2200%) ||- _RBL_MATCH: 297 (55.1000%) || ||- _RBL_MATCH_SPAMHAUS: 171 (57.5700%) ||- _RBL_MATCH_SPAMCOP : 126 (42.4200%) | |-DENIED : 905 (62.5000%) || ||- _RDNS_MISSING : 415 (45.8500%) ||- _RBL_MATCH: 446 (49.2800%) ||- _EARLYTALKER : 0 (0%) ||- _SENDER_NO_MX : 14 (1.5400%) ||- _TOO_MANY_RECIPIENTS : 0 (0%) ||- _UNQUALIFIED_RECIPIENT: 0 (0%) | |-Clamav : 4 (.2700%) | |- Phishing : 4 (100.%) |- Trojan: 0 (0%) On Tue, 2009-09-01 at 15:52 -0500, Sam Clippinger wrote: -Original Message- From: spamdyke-users-boun...@spamdyke.org [mailto:spamdyke-users-boun...@spamdyke.org] On Behalf Of Mirko Buffoni Sent: 01 September 2009 14:27 To: spamdyke users Subject: Re: [spamdyke-users] Spam Stats Goods average between 500 and 2000 daily. Figures are however pretty standard. Spamdyke filters out about 60k attempts daily. Here are yesterday stats: Good : 1025 = 0.68 % Unsure :183 = 0.12 % Virus : 62 = 0.04 % BAD Sender: 5114 = 3.40 % BAD Rcpt :212 = 0.14 % Pure SPAM : 45997 = 30.56 % SPAMMER : 97940 = 65.06 % | \.BLACKLISTED_KEYWORD : 29608 = 30.23 % \..DENIED_EARLYTALKER : 3 = 0.00 % \...DENIED_IP_IN_RDNS : 30447 = 31.09 % \DENIED_RBL_MATCH : 23268 = 23.76 % \.DENIED_SENDER_NO_MX : 13070 = 13.34 % \..DENIED_TOO_MANY_RECIPIENTS : 1 = 0.00 % \DENIED_UNQUALIFIED_RECIPIENT : 1 = 0.00 % \.TIMEOUT : 1542 = 1.57 % -- Total : 150533 = 100.00 % -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail + spamdyke + chkuser
Youri Kravatsky wrote: Hello, Eric! I investigated this problem more thoroughly and what can I say - qmail really REJECTS letters to non-existent users of ja-maica.ru (e.g. s...@ja-maica.ru), but ACCEPTS mails for users like (s...@www.ja-maica.ru) - it tries to work as MX server for all hosts at the networks. Probably, it's not the problem of chkuser/spamdyke, but the qmail itself? Can it be fixed to changing rcpthosts strings to something like @ja-maica.ru? Is your objective to accept or reject messages to the sub-domain(s)? Of course, to reject. Or, well, do not accept emails to hosts that are not included directly in rcpthosts file. At least to reject to all subdomains of our domains. What subdomains are you seeing besides @www. ? Subdomains of our domains. Mail that goes to domains that are not included to rcpthosts file is rejected. But mail to www.mydomain.com or mail.mydomain.com are accepted to deliver (to no avail, of course). Does your rcpthosts contain ja-maica.ru or .ja-maica.ru ? (or both?) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail + spamdyke + chkuser
Youri Kravatsky wrote: Hello, Eric! I investigated this problem more thoroughly and what can I say - qmail really REJECTS letters to non-existent users of ja-maica.ru (e.g. s...@ja-maica.ru), but ACCEPTS mails for users like (s...@www.ja-maica.ru) - it tries to work as MX server for all hosts at the networks. Probably, it's not the problem of chkuser/spamdyke, but the qmail itself? Can it be fixed to changing rcpthosts strings to something like @ja-maica.ru? Is your objective to accept or reject messages to the sub-domain(s)? Of course, to reject. Or, well, do not accept emails to hosts that are not included directly in rcpthosts file. At least to reject to all subdomains of our domains. What subdomains are you seeing besides @www. ? Subdomains of our domains. Mail that goes to domains that are not included to rcpthosts file is rejected. But mail to www.mydomain.com or mail.mydomain.com are accepted to deliver (to no avail, of course). Hey Youri, I think this is part of your problem: shu...@edwin:~$ host xyz.ja-maica.ru xyz.ja-maica.ru is an alias for www.ja-maica.ru. www.ja-maica.ru is an alias for www.dsite.ru. www.dsite.ru is an alias for dsite.ru. dsite.ru has address 89.108.80.21 dsite.ru mail is handled by 10 dsite.ru. shu...@edwin:~$ Any subdomain will find its way to your server. Is there any reason for the wildcard DNS record(s)? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail + spamdyke + chkuser
Youri V. Kravatsky wrote: Hello Eric, Saturday, September 5, 2009, 2:39:30 AM, you wrote: What subdomains are you seeing besides @www. ? Subdomains of our domains. Mail that goes to domains that are not included to rcpthosts file is rejected. But mail to www.mydomain.com or mail.mydomain.com are accepted to deliver (to no avail, of course). I think this is part of your problem: shu...@edwin:~$ host xyz.ja-maica.ru xyz.ja-maica.ru is an alias for www.ja-maica.ru. www.ja-maica.ru is an alias for www.dsite.ru. www.dsite.ru is an alias for dsite.ru. dsite.ru has address 89.108.80.21 dsite.ru mail is handled by 10 dsite.ru. Any subdomain will find its way to your server. Is there any reason for the wildcard DNS record(s)? Yes, it seems quite reasonable for www-hoster (I know, I know, it's not good practice to have www-carrier/database provider and mail server at one hardware, but we not always can change reality as we want). So in the case of any mistype/error users will access through HTTP any domain that is hosted succesfully. Really, the question is - why chkuser/qmail/spamdyke is accepting mail for subdomains, if they are not listed directly in rcpthosts? And how to stop it? Right now in rcpthosts is the string ja-maica.ru without . and/or @. Probably, I'll play with it at night, when the risk to lose some mail is minimal... Hey Youri, I gotta admit that I don't know off hand how subdomains are supposed to be handled by qmail. So I did some testing. The first test I sent to mys...@sub.mydomain.com. Interestingly enough, it was rejected because I have @mydomain.com in my blacklist_senders file. This is to prevent spamd where the sender address is spoofed with my domain. It works because all email for my domain is sent with authentication (a good practice), and authenticated users circumvent all spamdyke rules. I was curious about what happens without spamdyke doing this, so I did another test w/out having the blacklist entry. In the smtp log I got: 09-05 07:45:04 CHKUSER rejected relaying:... client not allowed to relay 09-05 07:45:05 DENIED_OTHER from: The message bounced back to the sender with: 553 sorry, that domain isn't in my list of allowed rcpthosts (#5.5.3 - chkuser) So chkuser recognized that subdom.mydomain.com was not a domain that my server is configured to receive mail for. This seems right to me. My chkuser version is 2.0.8, but that's the only difference I see. Your system should recognize that the subdomain isn't in your rcpthosts file. Are you certain that you don't have .ja-maica.com (with leading .) in your rcpthosts or morercpthosts files? If not, then I'd look closer into your chkuser implementation. Are you seeing any chkuser messages in your smtp log? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
[spamdyke-users] Whitelist methods
I am thinking that from a security standpoint, the preferred methods of whitelisting would be by: 1) rDNS 2) IP 3) sender simply because spoofing a sender is easiest and spoofing rDNS is the most difficult. Is this correct? Are there other considerations? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Feature request - whitelist SPF
I would think that SPF would be fairly easy to implement. There are libraries available (http://www.openspf.org/Implementations). I'm just looking at this as a more secure (and lazy) way to whitelist a domain. ;) Is there something I can do to help move this along? Sam Clippinger wrote: I don't see why this can't be done. Once SPF support is added, it should be pretty trivial to add a flag to control what spamdyke does with it. -- Sam Clippinger Eric Shubert wrote: Eric Shubert wrote: Hey Sam (et al), I just came across a situation where I wanted to whitelist a vendor (dyndns.com), so I requested their rDNS names. They cordially replied that they use various servers, and gave me their SPF record as reference. Then a little light went on. Spamdyke could do this for me. How about a spf-whitelist option, similar to the other whitelist options, that would read the SPF record for the sending domain and automatically whitelist according to the SPF rules found. This would effectively say, whitelist whatever servers are listed in the domain's SPF record - I'll trust their SPF record. I know this isn't trivial because of the variety of ways that senders can be specified in SPF, but I think the feature would be very useful. I would guess that most users would want to implement this only for certain domains. I'm not sure if turning it on globally would be ok to do or not. I'm thinking probably no, but it might be a nice option for some. Thoughts? I know you have SPF listed under TODO LATER in TODO.txt, but it's listed along with some other schemes which I believe are more involved to fully implement. I see this more of an enhancement of spamdyke's whitelisting capabilities than an outright SPF implementation. FWIW. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail + spamdyke + chkuser
Youri V. Kravatsky wrote: Hello Eric, Saturday, September 5, 2009, 7:43:00 PM, you wrote: The first test I sent to mys...@sub.mydomain.com. Interestingly enough, it was rejected because I have @mydomain.com in my blacklist_senders file. This is to prevent spamd where the sender address is spoofed with my domain. It works because all email for my domain is sent with authentication (a good practice), and authenticated users circumvent all spamdyke rules. Well, let's imagine, that you will send mail to thyself (or even more important, to the OTHER domain at your hosting), not through YOUR server, but through authenticated SMTP e.g. gmail.com, or through SMTP of his local internet provider (you know, cable providers blocks external SMTP servers access very freguently, and it is very reasonably, 'course). Then this mail will be definitely rejected, not being spam, but being inter-user communication. Right, as it should be. All email from my domain *is* (at least should be) sent through my server, where it is delivered locally. I can't imagine why I would want to send email from my domain and to my domain via any external server. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Qmail + spamdyke + chkuser
Eric Shubert wrote: Youri V. Kravatsky wrote: Hello Eric, Monday, September 7, 2009, 11:19:47 AM, you wrote: Right, as it should be. All email from my domain *is* (at least should be) sent through my server, where it is delivered locally. I can't imagine why I would want to send email from my domain and to my domain via any external server. Trust me, A LOT of providers block access to 25 port on the external networks INDEED. I do it myself in two organizations that I help to manage, 'cause I don't want to deal with permanent complains about spam that goes from infected computers. So, user with notebook has only one option - use local SMTP server in such kind of networks to deliver mail. Only one option that you can see. I have a VPN that road warriors use. ;) Or if you have no VPN, port 587 would suit the purpose. I haven't heard of an ISP blocking port 587, but it wouldn't surprise me if there are a few. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] working with /service/qmail-smtpd/run
Shepherd Nhongo wrote: Howdy ! I am running qmail according to qmailrocks guide and upgraded some servers according to John Simpson's site. Can someone with the following help me with showing me or sending me a modified /service/qmail-smtpd/run script? Look at my current /service/qmail-smtpd/run before spamdyke implementation. #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` LOCAL=`head -1 /var/qmail/control/me` if [ -z $QMAILDUID -o -z $NOFILESGID -o -z $MAXSMTPD -o -z $LOCAL ]; then echo QMAILDUID, NOFILESGID, MAXSMTPD, or LOCAL is unset in echo /var/qmail/supervise/qmail-smtpd/run exit 1 fi if [ ! -f /var/qmail/control/rcpthosts ]; then echo No /var/qmail/control/rcpthosts! echo Refusing to start SMTP listener because it'll create an open relay exit 1 fi exec /usr/local/bin/softlimit -m 3000 \ /usr/local/bin/tcpserver -v -R -l $LOCAL -x /etc/tcp.smtp.cdb -c $MAXSMTPD \ -u $QMAILDUID -g $NOFILESGID 0 smtp \ /var/qmail/bin/qmail-smtpd nhongo.co.cc http://nhongo.co.cc \ /home/vpopmail/bin/vchkpw /usr/bin/true 21 Regards, -- Shepherd Nhongo Here's the run file used with qmailtoaster (http://www.qmailtoaster.com) and spamdyke: #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` SPAMDYKE=/usr/local/bin/spamdyke SPAMDYKE_CONF=/etc/spamdyke/spamdyke.conf SMTPD=/var/qmail/bin/qmail-smtpd TCP_CDB=/etc/tcprules.d/tcp.smtp.cdb HOSTNAME=`hostname` VCHKPW=/home/vpopmail/bin/vchkpw REQUIRE_AUTH=0 exec /usr/bin/softlimit -m 2000 \ /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c $MAXSMTPD \ -u $QMAILDUID -g $NOFILESGID 0 smtp \ $SPAMDYKE --config-file $SPAMDYKE_CONF \ $SMTPD $VCHKPW /bin/true 21 -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Databases revisited
lenn...@wu-wien.ac.at wrote: Dear all, I have been reading up on the discussions on this list as well as the concerns about databases in the FAQ. Whilst I concur with most of the points wrt. to a fully fledged SQL database, I think that CDBs are ideally suited for the purposes of spamdyke. Sam states in the FAQ that speed, memory, concurrency, portability and availability are not a concern with CDBs and I agree, especially on the speed issue. After all, that was what the hash file format was designed for. That leaves accessibility and safety for CDBs. It is true that the database itself is in binary form (that is where the speed comes from), which means that they cannot be easily viewed and checked for errors. At the same time, they are read only and are usually generated from a plain text file as input. There is no reason to not have that text file sitting next to the actual database file, which means we have all the advantages of a plain text file plus the speed benefit of CDBs, which can be substantial for a lot of entries. The only additional step required (by the admin) would be to convert the text file into the CDB. We could also have the best of both worlds like this. Suppose we have this entry in the configuration file: recipient-blacklist-file=/etc/spamdyke/recipient-blacklist First, we look for a file with the name /etc/spamdyke/recipient-blacklist.cdb. If it exists, we assume it is a CDB version of /etc/spamdyke/recipient-blacklist and look up whatever we need there. If recipient-blacklist.cdb has an earlier modification time than recipient-blacklist (we get that for free anyway with a stat() on both files), we could help the admin by printing a warning that the CDB is probably out of date and read from recipient-blacklist instead. If recipient-blacklist.cdb does not exist, we use recipient-blacklist in ASCII format like before. Another version of this would be to have lots of new configuration options like: recipient-blacklist-file-cdb=/etc/spamdyke/recipient-blacklist.cdb That makes it possible to name the database file arbitrarily. If we want the safety checks like in the example above we could make it mandatory to name the ASCII input file for the CDB database file: recipient-blacklist-file=/etc/spamdyke/recipient-blacklist recipient-blacklist-file-cdb=/etc/spamdyke/recipient-blacklist.cdb That way all the fallbacks to ASCII plus warnings can be implemented at the cost of more configuration entries. What do you think? What problem specifically would this address? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Can't avoid spam check after auth
Jorge Minassian wrote: Hello, I am en *very* new user of spamdyke. I reached it looking for how to avoid Qmail (in a Plesk enviroment) mark own users as spam altough they get authenticated. I could install and get all working. But what I see is that incoming mail (from genuine users) still is getting high level of spam, inspected by spamassassin. Can any one give some tip to avoid this ?. Thank you very much. Jorge. I don't know about plesk. Qmail-Toaster can avoid this by having users submit via port 587 (the submission port) which does not invoke spamassassin. Does plesk have port 587 set up? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Databases revisited
Nice piece, Sam. In addition, the OS will likely have cached spamdyke's config file(s) anyhow, so I expect any real performance gain would be negligible. BL to me is that there are a host of other inefficiencies (pardon the pun) that would bring a mail server to its knees long before optimization of spamdyke's config files could provide any relief. Sam Clippinger wrote: Personally, I like the second option (adding options with -cdb for CDB files) rather than the first one (requiring a specific naming scheme). I've already implemented CDB support in the code for the next version, so spamdyke can read some of qmail's control files for recipient validation. Adding CDB support to other options wouldn't take much extra effort. The big question, of course, is whether it's worth it. I know DJB says CDB files are the bee's knees but I must say (after reading his docs, his source code and writing my own code for spamdyke) that I'm not impressed. I'm sure they're more efficient than text files for large amounts of data (hundreds of thousands of entries). But for small data sets (hundreds of entries) I don't believe they're any more efficient and for tiny data sets (ten entries) they are hugely wasteful. When you consider the additional headache of having to keep the CDB file in sync with the ASCII source, I really don't see the point. Of course I haven't benchmarked anything, so I could be way off base. DJB has a PhD and teaches computer science, I don't. He probably analyzed his hash functions to minimize collisions and compared operational complexities and so forth... academics do that kind of stuff for fun. In a nutshell, here's how a CDB file is accessed: Calculate hash Seek to position within CDB, read 64 bytes of data (primary hash table) A few more calculations Seek to another position within CDB, read another 64 bytes of data (secondary hash table) A few more calculations Seek to a third position within the CDB, read another 64 bytes of data (header entry) Compare the header entry to the desired data If it matches, seek to a fourth position within the CDB, read the data record If it does not match, go back to the secondary hash table and look in the next slot for your data. Repeat until your data is found. Except for the secondary hash table, which I don't see a need for, this describes a textbook hash table from freshman computer science classes. The seek/read operations are the most expensive operations (the math takes no time at all) because they require the program to wait for access to a spinning disk. If everything goes well and there are no hash collisions, reading a single entry from a CDB file requires 4 separate seek/read operations within the file. If things go badly and there are hash collisions, reading an entry from a CDB file may take many more read/seek operations (theoretically it could read the entire file). By comparison, when spamdyke reads a text file, it loads 64 KB at a time (if possible) and parses the lines in memory. This is a win when the file is small or the entry is near the beginning. It's a huge win when the file is tiny (like most /etc/tcp.smtp files). So I said all that to say this: I don't personally believe CDB files live up to the hype, nor do I believe they solve any real-world problems (they're still binary formats, they can't be shared between servers, etc) but if people want them I can support them. -- Sam Clippinger lenn...@wu-wien.ac.at wrote: Dear all, I have been reading up on the discussions on this list as well as the concerns about databases in the FAQ. Whilst I concur with most of the points wrt. to a fully fledged SQL database, I think that CDBs are ideally suited for the purposes of spamdyke. Sam states in the FAQ that speed, memory, concurrency, portability and availability are not a concern with CDBs and I agree, especially on the speed issue. After all, that was what the hash file format was designed for. That leaves accessibility and safety for CDBs. It is true that the database itself is in binary form (that is where the speed comes from), which means that they cannot be easily viewed and checked for errors. At the same time, they are read only and are usually generated from a plain text file as input. There is no reason to not have that text file sitting next to the actual database file, which means we have all the advantages of a plain text file plus the speed benefit of CDBs, which can be substantial for a lot of entries. The only additional step required (by the admin) would be to convert the text file into the CDB. We could also have the best of both worlds like this. Suppose we have this entry in the configuration file: recipient-blacklist-file=/etc/spamdyke/recipient-blacklist First, we look for a file with the name /etc/spamdyke/recipient-blacklist.cdb. If it
Re: [spamdyke-users] Databases revisited
BC wrote: Hi Sam - That is a pretty good synopsis of what he is doing. Doesn't he claim to find *any* sought after data in no more than 7 seeks? Maybe I misread that somewhere. :) My take on the below would be that if spamdyke remains a qmail-only spam blocker, then going with a cdb-based database would be okay (with the provisos you point out.) But if spamdyke is ultimately going to go mainstream (work for most any MTA), then I'd say pick the database you like the best. I'm thinking that no database might just be the best for this particular application (spamdyke). I don't know where people get the idea that databases provide better performance than a native filesystem. The database is implemented on top of a native filesystem after all. ;) If someone is really interested in speed, why not simply put spamdyke's config file(s) on a ram drive? Bucky On 10/22/2009 spamdyke-users-requ...@spamdyke.org wrote: So I said all that to say this: I don't personally believe CDB files live up to the hype, nor do I believe they solve any real-world problems (they're still binary formats, they can't be shared between servers, etc) but if people want them I can support them. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Databases revisited
Michael Colvin wrote: After looking into QMT, which has recipient validation built in, I'm not sure Spamdyke really needs it... The implementation in QMT allows for VPOPmail and non-VPOPmail qmail servers to easily validate recipients. If Spamdyke implemented a version based on cdb files, with VPOPmail servers, something would have to be put in place to build those cdb files from the database. Spamdyke is fantastic at what it does. I'm not sure that it needs to be complicated. Of course, as long as the validation is easy enough to disable, then I guess it wouldn't matter, and non-VPOPmail users could enable it and use the cdb files... If Spamdyke included the ability to validate against the VPOPmail database, I'm not sure it would be any more or less efficient than the patch that's included in QMT. Eric? My guess is that performance would be about the same whether spamdyke or chkuser does the validation. I don't see the issue as being performance related though. I'm more interested in having configuration options in a simple, manageable place. I'd like to see spamdyke handle whatever configuration variables are practical, even if spamdyke were to simply set an environment variable for some other code to pick up. The fewer number of patches to qmail source, the better. Which makes me wonder about chkuser. That patch is implemented in a non-invasive fashion, as most of the code sits outside of qmail proper. Most if not all of the chkuser configuration parameters can be altered with environment variables. Sam, have you looked at bringing chkuser functionality into the spamdyke realm? I would expect that you could probably find a way to integrate chkuser into spamdyke, eliminating the need for the chkuser patch to qmail. This would simply QMT a bit as well. Thanks for bringing this up Michael. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] spam tool recommendations
Jorge R. Constenla wrote: Hi, Spamdyke is great and blocks the 90% of Spam in our MXs servers. But we need another filter to block the rest of the spam. We receive more than 1M SMTP connections per day for many domains. Any recommendations ? Thanks in advance SpamAssassin (or DSpam) can filter much of what spamdyke doesn't catch. I'd always use spamdyke in conjunction with anything else though. Other spam filters are much more resource (cpu/ram) intensive. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
[spamdyke-users] Enhancement: require TLS when authenticating
While spamdyke can do both TLS and authentication, I don't see an option for requiring TLS when authenticating. I see smtp-auth-level settings of ondemand-encrypted and always-encrypted, but these -encrypted settings appear to refer to cram-md5, and they effect offering the protocol, not enforcing it. Also, my understanding is that cram-md5 is somewhat old-style, and less secure than TLS/SSL. It would be nice to be able to enforce from the server a policy of requiring TLS to be used with authentication, so that clients don't inadvertently send passwords in the clear. IOW, a setting that would check to be sure TLS was activated before processing any authentication command (possibly with the exception of cram-md5). It'd be great if this could work regardless of whether qmail or spamdyke is handling the encryption and/or authentication. Thanks Sam for all your great work on spamdyke. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Web app for configuring spamdyke
Michael Colvin wrote: I've not seen one. I've often thought of building something, and tying it into SpamAssassin also...A basic spam setting configuration page, that let users config options that are applicable to their accounts...Not so much for the Global settings...I still like doing those from a CLI... Maybe a toaster web app that configs the settings for the various applications on QMT? That would be certainly be sweet. I'm hoping that this spamdyke configuration may be a step in that direction. Any volunteers?? Michael J. Colvin NorCal Internet Services www.norcalisp.com -Original Message- From: spamdyke-users-boun...@spamdyke.org [mailto:spamdyke-users- boun...@spamdyke.org] On Behalf Of Eric Shubert Sent: Tuesday, November 03, 2009 3:23 PM To: spamdyke-users@spamdyke.org Subject: [spamdyke-users] Web app for configuring spamdyke Does anyone know of such a thing, or something someone might have worked on at some point? I know of someone who's interested in working on such an app, and am willing to put my 2 cents in. Does anyone know if such an app has ever been worked on? Would anyone else care to lend a hand? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Web app for configuring spamdyke
Thanks Ulrich. That's very helpful. I've passed the information on to our developer. Ulrich C. Manns wrote: Hi Eric, there is one: http://www.haggybear.de/en It works with Plesk and with the MySQL extension of spamdyke. It is also translated in english. See the attached screenshots. If you want a standalone solution please talk to Matthias (i...@haggybear.de). Regards, Ulrich Am 04.11.2009 um 00:22 schrieb Eric Shubert: Does anyone know of such a thing, or something someone might have worked on at some point? I know of someone who's interested in working on such an app, and am willing to put my 2 cents in. Does anyone know if such an app has ever been worked on? Would anyone else care to lend a hand? -- -Eric 'shubes' -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Enhancement: require TLS when authenticating
Sam Clippinger wrote: You could get close to this by using SMTPS (SMTP over SSL, so the entire connection is encrypted) and requiring authentication. The security would be as strong as SSL, which is pretty good. I'd rather stay away from that, as SMTPS is deprecated. While I like the idea of requiring encryption for authentication, I'm concerned that there's no way to communicate that requirement to the client. During SMTP, the server advertises its capabilities to the client, which is where authentication and TLS are offered. If TLS is started, the server is allowed to advertise a different set of capabilities to the client after encryption begins. But there's no way to say Authentication is not allowed only because TLS is not started; start TLS and you can authenticated. spamdyke would have to simply refuse to authenticate without TLS (and possibly reject all unauthenticated connections). I don't know the details of implementing such a feature, and don't really care how it's implemented (so long as it works!). I've learned since posting this request that there's a patch for qmail which causes it to refrain from advertising authentication until TLS is started. That is perhaps the correct way to do it. So, disregarding the support headaches for sysadmins who use such a feature, I could add a require-tls value to the smtp-auth-level option. That would be pretty easy. I'm wondering if this is really mutually exclusive of the other smtp-auth-level values. I guess requiring TLS would also imply the always behavior as it's presently defined. Perhaps adding always-require-tls would be a clearer value for this option. However, CRAM-MD5 is actually pretty secure. It's a challenge/response protocol, which means the password is never sent over the wire in any form. The server sends a challenge, which is just a big binary value (based on the server's name, the time and random numbers, so it's not predictable). Both the client and the server encrypt the challenge using the user's password as the encryption key (using the MD5 algorithm, hence the name). The client sends the result back to the server (the response), which the server compares to the value it calculated. If the two values match, the client and the server must have used the same password during the encryption, so the client is authenticated. Thus the security is as strong as MD5, which is pretty good. (IIRC, some researchers have demonstrated a few potential weaknesses in MD5 but nothing that would threaten this scenario in any practical way.) Thanks for this explanation Sam. Besides any concerns one might have about MD5's weakness though, CRAM-MD5 also requires the password(s) be stored in clear text, which is not acceptable in some situations, and is generally not a good practice from a security standpoint. -- Sam Clippinger Thanks as always, Sam. Spamdyke is unbelievably terrific! Eric Shubert wrote: While spamdyke can do both TLS and authentication, I don't see an option for requiring TLS when authenticating. I see smtp-auth-level settings of ondemand-encrypted and always-encrypted, but these -encrypted settings appear to refer to cram-md5, and they effect offering the protocol, not enforcing it. Also, my understanding is that cram-md5 is somewhat old-style, and less secure than TLS/SSL. It would be nice to be able to enforce from the server a policy of requiring TLS to be used with authentication, so that clients don't inadvertently send passwords in the clear. IOW, a setting that would check to be sure TLS was activated before processing any authentication command (possibly with the exception of cram-md5). It'd be great if this could work regardless of whether qmail or spamdyke is handling the encryption and/or authentication. Thanks Sam for all your great work on spamdyke. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Enhancement: log TLS indicator
Eric Shubert wrote: The todo file has a handfull of nice logging enhancements. Here's another. It'd be nice to have some indicator in the log of whether TLS was used on each session or not. This would allow easy verification that TLS is working on each message coming in. Thanks Sam. There's another aspect to this that Aleksander on the QMT list came across. He noticed that when spamdyke's doing the TLS encryption, there's no longer any indication in the message header that the message was encrypted as it was received. When qmail (patched with TLS) accepts a message using TLS, it notes that the message was received with encryption. Since spamdyke is passing the message in clear text to qmail, qmail no longer notes that TLS was used, even though spamdyke is dutifully decoding the encrypted session. The bottom line to this is that there's no practical way to audit that TLS is being used, or was used on a given message. I think this is a significant shortfall, while more so in some environments than others. Would it be possible for spamdyke to add a Received-spamdyke header of some sort that would indicate whether or not TLS was used? I imagine that other relevant information about spamdyke could be included, but I think Sam would have better ideas about this than I do. Thanks again Sam. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spam gets through even if its blacklisted
Will you post an example header of an email that passed spamdyke but was tagged as spamassassin? That would allow us to help you troubleshoot. Short of that, we can only speculate. Markus Thüer wrote: Hi, I got an interesting problem. I am running spamdyke on Plesk (8.04) for 18 Month now and it was working very nicely all the time. Since a few weeks a number of spams are getting through. But then they are identified and marked by spamassasin which is also running with plesk. But spamdyke is using the same blacklists as spamassasin. So they should be rejected before they reach spamassasin . The first thing I tried, was to update spamdyke for I was working with 3.1.8 So now I have the newest version but the behavior is still the same. I am not really an expert, but still I have to manage a server with 400 accounts and quite a bit of traffic. So if you could give me a hint where to look and how to find out why these mails are getting through I would be happy. Here my configuration: max-recipients=20 reject-empty-rdns reject-ip-in-cc-rdns reject-missing-sender-mx reject-unresolvable-rdns dns-blacklist-entry=zen.spamhaus.org dns-blacklist-entry=multi.uribl.com dns-blacklist-entry=bl.spamcop.net graylist-level=always graylist-dir=/var/qmail/spamdyke/greylist graylist-max-secs=1814400 graylist-min-secs=300 local-domains-file=/var/qmail/control/rcpthosts many thanks Markus ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] 64 bit
BC wrote: I'm looking to get WAY in over my head now. I'm considering going with a 64 bit version of the *nix OS I like (FreeBSD) with my next server install and am wondering if spamdyke (much less qmail) will work in such an OS? Short answer: yes with 64 bit, and I think so with FreeBSD. Understand that I am such a neophyte with all this that I might not understand the answers you offer. In fact I'll be using many of the responses not to give me info per se, but to offer me pointers to further reading. Fretfully, Bucky I recommend you have a look at http://qmailtoaster.com. It runs 64 bit with no problem, as you compile the source yourself (scripts are provided for easy installation). Unfortunately, it won't install on FreeBSD. qmailtoaster is ideal for neophytes such as yourself. The community list provides friendly help when you need it. If you decide to give QMT a try, I'd recommend using the CentOS 5 distro. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] hello
Arvydas wrote: hello, how come spamdyke does not block anonymo...@myhostna.me mailto:anonymo...@myhostna.me if i add it to blacklist_senders (it block all other domains, but this particular sender is not blocked..) sincerely, arvydas My guess would be that either a whitelist rule is being satisfied, or the messages is coming from an authenticated sender. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
[spamdyke-users] --config-test taking a while
I just upgraded spamdyke on one of my servers, and noticed that the --config-test was taking a considerable amount of time. I determined that it was the existing greylist tree that was making it take so long. I ran David Stiller's clean-up-script (posted on this list 10/08/2008) which reduced the tree from 97k to 60k entries (number of entries, not size). That helped a bit, but the --config-test still takes quite a while to complete. I'm planning on working on the script a bit this weekend to see if I can make some improvements to it. It appears to me from first glance that the script leaves a bunch of empty directories behind after cleaning up the empty files. I'll post anything I come up with. Thanks to David for the great start with this. If anyone has any additional information about cleaning up greylist trees, please let me know. Sam, do you have any thoughts about speeding up the --config-test run with relatively large greylist trees? P.S. Happy New Year everyone! -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Filter $SENDER == $RECIPIENT
Peter Palmreuther wrote: Hello, pardon me if my question has already been answered, but the overwhelming amount of information in documentation and FAQ maybe just made me not finding it. In this case I simple link to the answer would be appreciated. I'm getting a lot of spam mail with $SENDER being equal to $RECIPIENT. Maybe I'm dense, but I really don't remember having sent this messages to myself ;-) But: from time to to I do sent a reminder about some stuff to myself, so sometimes $SENDER == $RECIPIENT is OK. I know it would be a tough job for spamdyke knowing, when it's me and when it's not myself selnding this message, BUT: if I do it I do use SMTP-AUTH. So here's the question: is there any chance to configure spamdyke rejecting mails with $SENDER == $RECIPIENT or even better with DOMAIN($SENDER) in (RCPTHOSTS) UNLESS it's a authenticated SMTP connection? That's simple. Blacklist your domain using sender-blacklist-. See http://www.spamdyke.org/documentation/README.html#REJECTING_ADDRESSES and specify @yourdomain.com in the entry. You probably want to include all domains that are in your rcpthosts file. This is anti-intuitive, but it works. Since users of your domain always authenticate (or use a submission port 587), they always pass. Anything coming in that's not authenticated and appears to be from your domain is rejected. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Spamdykes timeouts, but Qmail delievers message - client resend message
Hans F. Nordhaug wrote: Hi - I'm new to the list but I have searched ;-) We are running Spamdyke 4.0.10 (as included in Qmail Toaster Plus) with idle-timeout-secs set to 60. One of my users recently got 30 duplicate messages (and wasn't happy). Looking at the logs, I see that Spamdyke indeed timed out but the messages were (scanned by simscan and) delievered by Qmail to the user. How can this be? I understand that the sender's e-mail client (Outlook most likely, but I haven't checked) must have thought that the messages weren't delievered and hence retried. I first thought that this was a known bug in SpamDyke - see discussion in the Spamdyke passes partial emails to qmail after timeout thread: http://www.mail-archive.com/spamdyke-users@spamdyke.org/msg00744.html But this bug was fixed long time ago. What exactly does Spamdyke do when it timeouts? If the the client keeps the connection open long after completing the message transfer and spamdyke does X when timing out, could that confuse the client so it resends? Regards, Hans Hey Hans, Need a little more info to tell for sure what's happening, like whether the messages was coming from an outside server, or from a local client. I'll go with your local client presumption, and also assume they're using Outlook. When Outlook submits a message, it must wait for the server to scan the message before the submission completes. If your server is under a heavy load when a user submits a particularly large email, this might take a couple minutes. The user doesn't notice this, as Outlook is sending from its Outbox in the background. Outlook times out after 1 minute by default, and if the server takes longer than that, Outlook obligingly sends the message again a little later. To fix this problem, increase Outlook's timeout setting. I do this on all Outlook clients. FWIW I also configure them to use port 587, so spamdyke doesn't come into play. ;) If this doesn't fix your problem, then I would guess that your server is under heavy load at times and is taking a long time to scan messages. If that's the case, you can remedy the situation by increasing the timeout setting to 300 or so. A better solution though would be to speed up scanning by putting the simscan working directory on a tmpfs (ram drive). Jake has a howto video on this at http://videos.qmailtoaster.com. I think that one's free, but I'm not sure. I expect that one or both of these solutions will solve your problem. P.S. If you're so inclined, please consider writing a FAQ about this on the qmailtoaster wiki. ;) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Fighting BCC spam
Marcin Orlowski wrote: Hi, Apologies for partially off-topic thread, however not spamdyke but qmail/spam related. I recently noticed increased number of what I call BCC Spam. It looks like From: is external, To: is local user (so mail is accepted) but there're also external BCC: recepipients. To my understanding (which aparently seems incorrect?) qmail should only bother BCC is From is local. But it does not and happily spends this spam out to BCC: targets. I did not investigate deeply yet, but as a quick solution I could probably play with qmail-inject.c to tweak qmail a bit but I do not like this approach right now, so I wonder if anyone else faced this issue already and manage to solve it? Regards, This sounds me like like it's coming from a host that's infected with malware. The From: address may be external, but what's the IP address of the sender? Also, is it coming from an authenticated user's account? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
[spamdyke-users] graylist cleanup script
Back in Oct'08 David Stiller posted a nice script for cleaning up
graylist trees. Unfortunately, it doesn't work as well as it might with
v4.0 graylist trees, leaving empty directories behind.
I've taken the liberty to use it as a base for a new script I've written
for v4 (and v3) spamdyke graylists. The attached qtp-prune-graylist
script is part of the qmailtoaster-plus package
(http://qtp.qmailtoaster.com). The script can be obtained there on its
own without having to install the whole QTP package by browsing the svn
repository, and selecting Original format at the bottom of the page.
It can be run stand alone, and requires only a spamdyke configuration
file to operate.
The script obtains the graylist location and duration parameters from
the /etc/spamdyke/spamdyke.conf file by default. If you have a spamdyke
configuration file in another location, simply edit the script to point
to that location. If you specify these configuration parameters in
spamdyke's command line, you'll need to modify the script appropriately.
The script handles any number of domains and accounts. A recent run on a
host that had been running spamdyke v3.x for quite a while produced:
qtp-prune-graylist total - 17 domains processed
qtp-prune-graylist total - 1134404 entries found
qtp-prune-graylist total - 1128058 entries removed
qtp-prune-graylist total - 0 empty directories removed
qtp-prune-graylist total - 6740 graylisting entries remain
This host had run out of inodes before the script was run. Afterwards
12% of inodes were in use.
Counts by domain are shown as the script runs. There is an -s flag
available for silent running (think cron).
You'll probably notice that after running the script, the --config-test
option of spamdyke runs a bit faster, as it walks the entire graylist tree.
There is no option for trial run like in the former script. That seemed
like a waste to me.
Please use the ticket system provided at http://qtp.qmailtoaster.com to
report any bugs or make any enhancement requests you might have.
--
-Eric 'shubes'
#!/bin/bash
#
# Copyright (C) 2010 Eric Shubert e...@shubes.net
#
# This script removes old/expired entries in spamdyke's graylist tree.
#
# Original script written by David Stiller , posted on the spamdyke list.
# Enhanced by shubes to obtain parameters from spamdyke configuration,
# and do a more thorough job of pruning.
#
#
# change log
# 01/26/10 shubes - created from David Stiller's greylist-clean.sh script
#
# This should be the only thing you might need to change
# Location of spamdyke configuration file
sdconf=/etc/spamdyke/spamdyke.conf
#
# check/obtain parameter values
#
a2_check_parameters(){
if [ ! -f $sdconf ]; then
echo $me - config file \$sdconf\ does not exist
exit 1
fi
gldir=$(q21_get_spamdyke_parm graylist-dir)
glmax=$(q21_get_spamdyke_parm graylist-max-secs)
if [ ! $silent ]; then
echo $me processing graylist tree at $gldir ...
echo $me pruning entries older than $glmax seconds ...
fi
}
#
# get a spamdyke configuration parameter
#
q21_get_spamdyke_parm(){
sdparm=$(grep $1= $sdconf)
echo ${sdparm#$1=}
}
#
# process each domain in the graylist tree
#
a5_process_domain(){
domname=${dompath##*/}
if [ ! $silent ]; then
echo $me processing domain $domname ...
fi
domtot=$(q51_count_graylist_entries)
if [ ! $silent ]; then
echo $me $domname - $domtot entries found
fi
# delete files that are expired
domdlf=$(find $dompath -type f -mmin +$[$glmax/60] -exec rm {} \; -print | wc
-l)
# delete empty directories
domdld=$(find $dompath -depth -mindepth 2 -type d -empty -exec rmdir {} \;
-print | wc -l)
domrem=$(q51_count_graylist_entries)
if [ ! $silent ]; then
echo $me $domname - $domdlf entries removed
echo $me $domname - $domdld empty directories removed
echo $me $domname - $domrem graylisting entries remain
fi
graydom=$[$graydom+1]
graytot=$[$graytot+$domtot]
graydlf=$[$graydlf+$domdlf]
graydld=$[$graydld+$domdld]
grayrem=$[$grayrem+$domrem]
}
#
# count the number of files (entries) in the graylist tree
#
q51_count_graylist_entries(){
echo $(find $dompath -type f | wc -l)
}
#
# main execution begins here
Re: [spamdyke-users] new version of spamdyke?
nightduke wrote: Hi i would like to know when will we released a new version of spamdyke, i still using version of 2008. Thanks 4.0.10 was released 12/17/08. I'm not aware of any bugs since then. Only Sam can say for sure when a new release will be coming. Are you looking for something in particular? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] new version of spamdyke?
Which graylisting features are you looking for specificallly? Have you run the qtp-prune-graylist script I posted recently? Perhaps that will solve your performance issue? If not, what are you seeing that leads you to believe there is a problem with performance? nightduke wrote: More featues with graylisting.Better perfomance. 2010/2/10 Eric Shubert e...@shubes.net: nightduke wrote: Hi i would like to know when will we released a new version of spamdyke, i still using version of 2008. Thanks 4.0.10 was released 12/17/08. I'm not aware of any bugs since then. Only Sam can say for sure when a new release will be coming. Are you looking for something in particular? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] new version of spamdyke?
Jorge R. Constenla wrote: The SpamDyke works great! without bugs. But is Very usefull (Excellent), if you can set some features per domain. Two Level to filter SPAM - General Level for all domains (the actual level) - And add a Domain Level Filter with features like: blacklist and whitelist lists (sender-blacklist, , etc ...) per domain. I believe this can be done, beginning with version 4. See http://www.spamdyke.org/documentation/README.html#CONFIGURATION_DIR -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] new version of spamdyke?
nightduke wrote: More features with graylisting,fast enable graylisting, I don't know what you mean by this. Perhaps Sam does. gui for spamdyke, There has been talk about this, and I believe someone has written something for this. I don't recall off hand though. You'll need to do some searching to find it. log level,log analyser, warnings, error, I think you need to be more specific here about what you'd like to see. dspam options to integrate with dspam. I don't know what you have in mind here. Please describe more thoroughly. I mean those features, will be great, subdomain options too... In my modest opinion. Of course. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
