[spamdyke-users] Feature request
Hello, I really think mail servers shoud reject connections from IP without rdns, I was tryng to impose it to my customers but I think I must desist for too much good mail being rejected. I't could be useful to have a per domain reject-empty-rdns, to gradually implement this feature. Just my 0.02$ Ciao Paolo ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Feature request
Hi, sorry I was not clear. I mean to enable reject-empty-rdns only for some recipients domains . Thank you Ciao Paolo Il giorno 14 apr 2008, alle ore 10:30, Andras Korn ha scritto: On Mon, Apr 14, 2008 at 09:49:11AM +0200, Paolo wrote: I really think mail servers shoud reject connections from IP without rdns, I was tryng to impose it to my customers but I think I must desist for too much good mail being rejected. I't could be useful to have a per domain reject-empty-rdns, to gradually implement this feature. How do you mean, per-domain? If the IP has no rdns, there is no domain. Andras -- Andras Korn korn at chardonnay.math.bme.hu http://chardonnay.math.bme.hu/~korn/ QOTD: Bad command or file name! Go stand in the corner. ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Feature request
The next version of spamdyke already includes this feature. -- Sam Clippinger Paolo wrote: Hi, sorry I was not clear. I mean to enable reject-empty-rdns only for some recipients domains . Thank you Ciao Paolo Il giorno 14 apr 2008, alle ore 10:30, Andras Korn ha scritto: On Mon, Apr 14, 2008 at 09:49:11AM +0200, Paolo wrote: I really think mail servers shoud reject connections from IP without rdns, I was tryng to impose it to my customers but I think I must desist for too much good mail being rejected. I't could be useful to have a per domain reject-empty-rdns, to gradually implement this feature. How do you mean, per-domain? If the IP has no rdns, there is no domain. Andras -- Andras Korn korn at chardonnay.math.bme.hu http://chardonnay.math.bme.hu/~korn/ QOTD: Bad command or file name! Go stand in the corner. ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL?
Andras Korn wrote: On Sun, Apr 13, 2008 at 02:55:16PM -0500, Sam Clippinger wrote: Most qmail servers run a stock version of qmail-smtpd, which will only reject recipients for relaying. They shouldn't, as stock qmail is liable to causing backscatter. No self-respecting admin should run qmail as distributed by DJB today. Mail to bogus recipients must not be accepted and then bounced later. Some mechanism must be in place to ensure that mail to bogus recipients is never accepted at all. I agree. But regardless of what /should/ be the case, the fact is that most qmail servers run the stock version of qmail-smtpd. I can't justify making a change that will make spamdyke less efficient for the majority and only slightly more efficient for the minority. I think you should always wait for RCPT TO, even if it's not necessary for whitelist decisions, because then you can log whose mail you're rejecting. rblsmtpd's inability to do this is one of its major shortcomings. spamdyke does always wait for RCPT, so that it can log the recipient. But it does not keep qmail running the entire time, if there is no chance the message will be accepted. If a recipient whitelist is not in use, there's no reason to wait. I suspect we're debating fractional efficiencies here anyway -- I've never benchmarked either scenario. Well, fwiw, I just ran a quick test: querying the handful of RBLs I have configured in parallel takes about 10 seconds (as long as it takes for the slowest of them to reply). Rejecting a mail based on the local list of valid recipients takes a good deal less than a second. Of course local file accesses are faster than DNS queries. That's not what I meant. To find real numbers, you would have to consider how many connections are accepted, how many are rejected and for what reasons. Then look at the popularity of different spamdyke features and specifically the popularity of different DNS RBLs. Use all that to find out what percentage of rejected connections could avoid the DNS queries due to local tests. Lastly, find a way to evaluate the real cost (wall time, server load and network load) of spamdyke's DNS queries versus the additional load generated by passing the extra SMTP traffic to qmail. That last step is the part I don't know how to measure. If all of those numbers were available, my instinct says the advantage of your proposed change would be very small at best. -- Sam Clippinger ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL?
Yes, this is certainly possible. Right now spamdyke identifies the RBL in its message to the remote server but not in the logs. Good idea! What would be a good way to log this information (preferably without breaking existing scripts)? I'm thinking as I type here, but spamdyke already follows the rejection reason with parenthesis (when the log level is high enough) to indicate which file/line matched for file-based filters... perhaps the same could be done for RBLs/RHSBLs. Something like this: DENIED_RBL_MATCH(rbl.example.com) As for reordering the RBLs to put the often-matched ones first, the next version of spamdyke will make that less necessary. By default, it will query all RBLs simultaneously, regardless of their order. (That behavior can be prevented with a new flag -- ordering would be important in that case.) -- Sam Clippinger Michael Colvin wrote: To find real numbers, you would have to consider how many connections are accepted, how many are rejected and for what reasons. Then look at the popularity of different spamdyke features and specifically the popularity of different DNS RBLs. Use all that to find out what percentage of rejected connections could avoid the DNS queries due to local tests. Along those lines, is it possible, or can it be possible, to have spamdyke's logs indicate which DNS RBL caused a message to be rejected? I'm assuming that once a reason for rejection is found, IE, the IP is listed in a particular RBL, further tests against other RBL's in the list are not performed? Knowing, statistically, which ones have a higher rejection rate, and queuing those first in the list of RBLS might save some time. Or course, multiple RBLS could reject the same message, and the one first in line would have the higher percentage, but this would give us a way to move them around and check the results... Just a thought from a newbie to spamdyke. BTW, I LOVE Spamdyke! What a difference it has made in my system's ability to filter spam and save resources! It's a God send! Mike ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL?
On Mon, Apr 14, 2008 at 09:40:51AM -0500, Sam Clippinger wrote: Andras Korn wrote: On Sun, Apr 13, 2008 at 02:55:16PM -0500, Sam Clippinger wrote: Most qmail servers run a stock version of qmail-smtpd, which will only reject recipients for relaying. They shouldn't, as stock qmail is liable to causing backscatter. No self-respecting admin should run qmail as distributed by DJB today. Mail to bogus recipients must not be accepted and then bounced later. Some mechanism must be in place to ensure that mail to bogus recipients is never accepted at all. I agree. But regardless of what /should/ be the case, the fact is that most qmail servers run the stock version of qmail-smtpd. I can't justify making a change that will make spamdyke less efficient for the majority and only slightly more efficient for the minority. You yourself said that the decreased efficiency, if any, would be marginal. This behaviour could even be configurable: delay-dns-blacklis-checks=1|0 or similar, defaulting to off if you're worried about efficiency. To find real numbers, you would have to consider how many connections are accepted, how many are rejected and for what reasons. Then look at the popularity of different spamdyke features and specifically the popularity of different DNS RBLs. Use all that to find out what percentage of rejected connections could avoid the DNS queries due to local tests. I can come up with local figures, but knowing globally what features of spamdyke are used how often is probably impossible. Lastly, find a way to evaluate the real cost (wall time, server load and network load) of spamdyke's DNS queries versus the additional load generated by passing the extra SMTP traffic to qmail. I can't imagine this latter additional load as being nontrivial, but it could be measured to some extent using strace -c. If all of those numbers were available, my instinct says the advantage of your proposed change would be very small at best. This is still ignoring the unnecessary load on the RBL DNS servers (most of us are using them for free, yet someone must pay for their maintenance and bandwidth, so let's not be wasteful). Also, I still think that in the case of email service, saving wall time (i.e. reducing latency) is more beneficial than saving CPU time (probably a minuscule amount of CPU time, at that). I find a net gain of 9 seconds per message with a single bogus recipient hard to ignore. Andras -- Andras Korn korn at chardonnay.math.bme.hu http://chardonnay.math.bme.hu/~korn/ QOTD: History will record it. I know it because I'll write it myself. ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL?
Great! Of course, this feature could also be used to determine if a specific RBL is causing to many false-positives too... Running all the checks simultaneously certainly will negate the need to order them in any specific order and should make the overall process that much faster, especially if you're using multiple RBL's. What kind of effect will that have on server load? Many RBL lookups sequencially versus many RBL lookups simultaneously? Seems like the process might be faster, but will take more resources on the server? Which would likely mean a basic Push with the net result being faster handling of the session? Thanks again! Mike -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sam Clippinger Sent: Monday, April 14, 2008 9:40 AM To: spamdyke users Subject: Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL? Yes, this is certainly possible. Right now spamdyke identifies the RBL in its message to the remote server but not in the logs. Good idea! What would be a good way to log this information (preferably without breaking existing scripts)? I'm thinking as I type here, but spamdyke already follows the rejection reason with parenthesis (when the log level is high enough) to indicate which file/line matched for file-based filters... perhaps the same could be done for RBLs/RHSBLs. Something like this: DENIED_RBL_MATCH(rbl.example.com) As for reordering the RBLs to put the often-matched ones first, the next version of spamdyke will make that less necessary. By default, it will query all RBLs simultaneously, regardless of their order. (That behavior can be prevented with a new flag -- ordering would be important in that case.) -- Sam Clippinger Michael Colvin wrote: To find real numbers, you would have to consider how many connections are accepted, how many are rejected and for what reasons. Then look at the popularity of different spamdyke features and specifically the popularity of different DNS RBLs. Use all that to find out what percentage of rejected connections could avoid the DNS queries due to local tests. Along those lines, is it possible, or can it be possible, to have spamdyke's logs indicate which DNS RBL caused a message to be rejected? I'm assuming that once a reason for rejection is found, IE, the IP is listed in a particular RBL, further tests against other RBL's in the list are not performed? Knowing, statistically, which ones have a higher rejection rate, and queuing those first in the list of RBLS might save some time. Or course, multiple RBLS could reject the same message, and the one first in line would have the higher percentage, but this would give us a way to move them around and check the results... Just a thought from a newbie to spamdyke. BTW, I LOVE Spamdyke! What a difference it has made in my system's ability to filter spam and save resources! It's a God send! Mike ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL?
The aggressive DNS queries will definitely increase the momentary load on the DNS servers, because they will get a burst of simultaneous queries each time a remote server connects. However, the overall load won't go up because the long-term rate of DNS queries is determined by the rate of SMTP connections. I believe most sites will only notice an increase in spamdyke's speed with no new load on their DNS servers. The new DNS behavior is configurable though, including an option to return to the behavior used by the standard system resolver. That way, if the DNS servers are having trouble, spamdyke can be made less demanding. -- Sam Clippinger Michael Colvin wrote: Great! Of course, this feature could also be used to determine if a specific RBL is causing to many false-positives too... Running all the checks simultaneously certainly will negate the need to order them in any specific order and should make the overall process that much faster, especially if you're using multiple RBL's. What kind of effect will that have on server load? Many RBL lookups sequencially versus many RBL lookups simultaneously? Seems like the process might be faster, but will take more resources on the server? Which would likely mean a basic Push with the net result being faster handling of the session? Thanks again! Mike -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sam Clippinger Sent: Monday, April 14, 2008 9:40 AM To: spamdyke users Subject: Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL? Yes, this is certainly possible. Right now spamdyke identifies the RBL in its message to the remote server but not in the logs. Good idea! What would be a good way to log this information (preferably without breaking existing scripts)? I'm thinking as I type here, but spamdyke already follows the rejection reason with parenthesis (when the log level is high enough) to indicate which file/line matched for file-based filters... perhaps the same could be done for RBLs/RHSBLs. Something like this: DENIED_RBL_MATCH(rbl.example.com) As for reordering the RBLs to put the often-matched ones first, the next version of spamdyke will make that less necessary. By default, it will query all RBLs simultaneously, regardless of their order. (That behavior can be prevented with a new flag -- ordering would be important in that case.) -- Sam Clippinger Michael Colvin wrote: To find real numbers, you would have to consider how many connections are accepted, how many are rejected and for what reasons. Then look at the popularity of different spamdyke features and specifically the popularity of different DNS RBLs. Use all that to find out what percentage of rejected connections could avoid the DNS queries due to local tests. Along those lines, is it possible, or can it be possible, to have spamdyke's logs indicate which DNS RBL caused a message to be rejected? I'm assuming that once a reason for rejection is found, IE, the IP is listed in a particular RBL, further tests against other RBL's in the list are not performed? Knowing, statistically, which ones have a higher rejection rate, and queuing those first in the list of RBLS might save some time. Or course, multiple RBLS could reject the same message, and the one first in line would have the higher percentage, but this would give us a way to move them around and check the results... Just a thought from a newbie to spamdyke. BTW, I LOVE Spamdyke! What a difference it has made in my system's ability to filter spam and save resources! It's a God send! Mike ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] let qmail decide if it accepts a recipient before doing RHSBL?
I like having specific RBLs logged. I just installed spamdyke on a few qmail-toasters yesterday (replacing rblsmtpd), and was going to as about this. Michael beat me to it! ;) If simultaneous queries are being done, can all RBLs that match be logged? Perhaps a comma separated list within parenthesis. This would make it possible to gather stats on the effectiveness of the RBLs being used. Sam Clippinger wrote: Yes, this is certainly possible. Right now spamdyke identifies the RBL in its message to the remote server but not in the logs. Good idea! What would be a good way to log this information (preferably without breaking existing scripts)? I'm thinking as I type here, but spamdyke already follows the rejection reason with parenthesis (when the log level is high enough) to indicate which file/line matched for file-based filters... perhaps the same could be done for RBLs/RHSBLs. Something like this: DENIED_RBL_MATCH(rbl.example.com) As for reordering the RBLs to put the often-matched ones first, the next version of spamdyke will make that less necessary. By default, it will query all RBLs simultaneously, regardless of their order. (That behavior can be prevented with a new flag -- ordering would be important in that case.) -- Sam Clippinger Michael Colvin wrote: To find real numbers, you would have to consider how many connections are accepted, how many are rejected and for what reasons. Then look at the popularity of different spamdyke features and specifically the popularity of different DNS RBLs. Use all that to find out what percentage of rejected connections could avoid the DNS queries due to local tests. Along those lines, is it possible, or can it be possible, to have spamdyke's logs indicate which DNS RBL caused a message to be rejected? I'm assuming that once a reason for rejection is found, IE, the IP is listed in a particular RBL, further tests against other RBL's in the list are not performed? Knowing, statistically, which ones have a higher rejection rate, and queuing those first in the list of RBLS might save some time. Or course, multiple RBLS could reject the same message, and the one first in line would have the higher percentage, but this would give us a way to move them around and check the results... Just a thought from a newbie to spamdyke. BTW, I LOVE Spamdyke! What a difference it has made in my system's ability to filter spam and save resources! It's a God send! Mike -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
