Re: R: R: R: Relay Checker Plugin (code review please?)

[Nix]

On 1 Nov 2006, Andreas Pettersson stated:

 Steven Dickenson wrote:
 I can't agree with this.  Many small businesses in the US get just  these 
 kind of static connections from broadband ISPs.
 Comcast, for  example, has all of their static customers using rDNS that 
 would fail  your tests, and they refuse to set up a
 custom PTR record or delegate  the record to someone else.

 I disagree on your disagreement. This is my opinion: If you don't have 
 control over your rDNS, do NOT run any mail server, unless
 you relay all outbound mail through a server at your ISP.

What if you don't *have* a server at your ISP that you can relay your
mail through, because your ISP expects you to send mail directly from
your own mailserver?

What if your ISP provides a server but it is horrifically unreliable?

 Most of these static customers are legitimate business networks
 running their own mail server, and have neither the need nor desire
 to relay their mail through Comcast's SMTP servers.  I think your
 general idea is very good, but you're reaching a little too far with
 this one.

 'No need nor desire', that's not really any good excuse. Use a relay
 or find your mail rejected, I'd say.

Charming. They're not spammers, but you want to punish them as if they
were, because reality makes your tests too complicated.

-- 
`When we are born we have plenty of Hydrogen but as we age our
 Hydrogen pool becomes depleted.'

Re: R: Relay Checker Plugin (code review please?)

[Nix]

On 31 Oct 2006, John Rudd verbalised:
 And, while I may be a little unyielding wrt to people whose ISPs are
 like Telecom Italia, I'm not unsympathetic.  I think, in this case, if
 Italy did get mass quarantined by the rest of the world, it might
 cause enough of an uproar to force Telecom Italia to change its
 practices and allow custom RDNS.

SPEWS tried that.

It didn't really work :(

-- 
`When we are born we have plenty of Hydrogen but as we age our
 Hydrogen pool becomes depleted.'

Re: R: R: R: Relay Checker Plugin (code review please?)

[John Rudd]

Nix wrote:

On 1 Nov 2006, Andreas Pettersson stated:


Steven Dickenson wrote:

I can't agree with this.  Many small businesses in the US get just  these kind 
of static connections from broadband ISPs.
Comcast, for  example, has all of their static customers using rDNS that would 
fail  your tests, and they refuse to set up a
custom PTR record or delegate  the record to someone else.

I disagree on your disagreement. This is my opinion: If you don't have control 
over your rDNS, do NOT run any mail server, unless
you relay all outbound mail through a server at your ISP.


What if you don't *have* a server at your ISP that you can relay your
mail through, because your ISP expects you to send mail directly from
your own mailserver?

What if your ISP provides a server but it is horrifically unreliable?


In those two cases:  Go to a service that hosts web/email servers under 
your custom domain, and relay through your own hosted server.  They 
exist.  Some of them aren't expensive at all.




Most of these static customers are legitimate business networks
running their own mail server, and have neither the need nor desire
to relay their mail through Comcast's SMTP servers.  I think your
general idea is very good, but you're reaching a little too far with
this one.

'No need nor desire', that's not really any good excuse. Use a relay
or find your mail rejected, I'd say.


Charming. They're not spammers, but you want to punish them as if they
were, because reality makes your tests too complicated.


Punishing them would be blocking them outright.  Quarantining them is 
merely recognizing that they haven't obtained a class of service that 
would indicate a well configured and well maintained email server, as 
opposed to a fly-by-night or rinky-dink operation running on a 
bottom-feeder ISP's network ... characteristics that would indicate that 
the ISP is careless and not diligent, or that their customers have no 
care nor concern about their quality of service, either one making it 
more likely that I am dealing with a spambot or an open-relay.  So, I 
make them jump through an extra hoop in order to get through to me. 
That extra hoop is either a) going through my quarantine process, or b) 
paying for a hosted service with proper RDNS.  Either don't look like 
part of a botnet or accept being in my quarantine all of the time.


It is not my obligation to accept nor view every email that hits my 
server.  It is my prerogative to establish whatever hurdles I want to in 
getting through to my email inbox.  It's not punishment, as punishment 
implies that I am removing your rights or privileges ... which doesn't 
apply, because you have no rights nor privileges with regard to my 
inbox.   And, the fact is, the people you're trying to raise as a 
counter-case, are not only such a minority that they aren't on my radar, 
they're such a minority that they don't exist at all in my 15 months of 
experience in doing these checks.


Every sender which has connected to my machines, without being on my own 
subnet, nor performing SMTP-AUTH, and without passing these checks, has 
been sending spam or a virus.  Without exception.



I realize that not everyone else is going to have the same experience 
with their email traffic that I do, which is why I'm making the plugin 
ever more flexible.  First, I set a preference for just skipping some 
tests.  Then, last night, I removed the hard-coded dynhostname and 
clienthostname checks.  Now there's a keyword check, with the keywords 
used being supplied in the cf file.  So, each site can set their own 
keyword requirement ... or leave it blank and skip that check entirely. 
(I haven't released this new code yet)

Re: R: R: R: Relay Checker Plugin (code review please?)

[Billy Huddleston]

I like the ability to give individual scores to the various tests...  Per my 
patch..  Allows for minor tweaking for each kind of issue.


- Original Message - 
From: John Rudd [EMAIL PROTECTED]

To: Nix [EMAIL PROTECTED]
Cc: Andreas Pettersson [EMAIL PROTECTED]; Steven Dickenson 
[EMAIL PROTECTED]; Giampaolo Tomassoni [EMAIL PROTECTED]; 
users@spamassassin.apache.org

Sent: Sunday, November 05, 2006 1:57 PM
Subject: Re: R: R: R: Relay Checker Plugin (code review please?)



Nix wrote:

On 1 Nov 2006, Andreas Pettersson stated:


Steven Dickenson wrote:
I can't agree with this.  Many small businesses in the US get just 
these kind of static connections from broadband ISPs.
Comcast, for  example, has all of their static customers using rDNS 
that would fail  your tests, and they refuse to set up a

custom PTR record or delegate  the record to someone else.
I disagree on your disagreement. This is my opinion: If you don't have 
control over your rDNS, do NOT run any mail server, unless

you relay all outbound mail through a server at your ISP.


What if you don't *have* a server at your ISP that you can relay your
mail through, because your ISP expects you to send mail directly from
your own mailserver?

What if your ISP provides a server but it is horrifically unreliable?


In those two cases:  Go to a service that hosts web/email servers under 
your custom domain, and relay through your own hosted server.  They exist. 
Some of them aren't expensive at all.




Most of these static customers are legitimate business networks
running their own mail server, and have neither the need nor desire
to relay their mail through Comcast's SMTP servers.  I think your
general idea is very good, but you're reaching a little too far with
this one.

'No need nor desire', that's not really any good excuse. Use a relay
or find your mail rejected, I'd say.


Charming. They're not spammers, but you want to punish them as if they
were, because reality makes your tests too complicated.


Punishing them would be blocking them outright.  Quarantining them is 
merely recognizing that they haven't obtained a class of service that 
would indicate a well configured and well maintained email server, as 
opposed to a fly-by-night or rinky-dink operation running on a 
bottom-feeder ISP's network ... characteristics that would indicate that 
the ISP is careless and not diligent, or that their customers have no care 
nor concern about their quality of service, either one making it more 
likely that I am dealing with a spambot or an open-relay.  So, I make them 
jump through an extra hoop in order to get through to me. That extra hoop 
is either a) going through my quarantine process, or b) paying for a 
hosted service with proper RDNS.  Either don't look like part of a 
botnet or accept being in my quarantine all of the time.


It is not my obligation to accept nor view every email that hits my 
server.  It is my prerogative to establish whatever hurdles I want to in 
getting through to my email inbox.  It's not punishment, as punishment 
implies that I am removing your rights or privileges ... which doesn't 
apply, because you have no rights nor privileges with regard to my inbox. 
And, the fact is, the people you're trying to raise as a counter-case, are 
not only such a minority that they aren't on my radar, they're such a 
minority that they don't exist at all in my 15 months of experience in 
doing these checks.


Every sender which has connected to my machines, without being on my own 
subnet, nor performing SMTP-AUTH, and without passing these checks, has 
been sending spam or a virus.  Without exception.



I realize that not everyone else is going to have the same experience with 
their email traffic that I do, which is why I'm making the plugin ever 
more flexible.  First, I set a preference for just skipping some tests. 
Then, last night, I removed the hard-coded dynhostname and clienthostname 
checks.  Now there's a keyword check, with the keywords used being 
supplied in the cf file.  So, each site can set their own keyword 
requirement ... or leave it blank and skip that check entirely. (I haven't 
released this new code yet)

R: R: R: R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

  Most of these static customers are  legitimate business networks 
  running their own mail server, and have  neither the need nor desire 
  to relay their mail through Comcast's  SMTP servers.  I think your 
  general idea is very good, but you're  reaching a little too far with 
  this one.
 
 'No need nor desire', that's not really any good excuse. Use a relay or 
 find your mail rejected, I'd say.

He doesn't need any excuse. From his point of view (and from mine too), you 
would need it. There is no RFC stating that mail not conforming to your 
requirements have to be dropped.

I well understand adding reasonable penalty scrores to them, not stopping them 
at once.

However, the customer is your. So...

---
Giampaolo Tomassoni - IT Consultant
Piazza VIII Aprile 1948, 4
I-53044 Chiusi (SI) - Italy
Ph: +39-0578-21100

MAI inviare una e-mail a:
NEVER send an e-mail to:
 [EMAIL PROTECTED]

 
 -- 
 Andreas
 
 

Re: Relay Checker Plugin (code review please?)

[Billy Huddleston]

I've attached the patch file this time.. give it a go..

Use this command to patch your file.

patch  RelayChecker.patch

and it should work..  This is just the patch for the .pm file.. the other 
one was simply adding in the default score values..


Thanks, Billy

- Original Message - 
From: Dylan Bouterse [EMAIL PROTECTED]

To: users@spamassassin.apache.org
Sent: Wednesday, November 01, 2006 11:28 PM
Subject: RE: Relay Checker Plugin (code review please?)


I did a couple of times. :(


-Original Message-
From: Billy Huddleston [mailto:[EMAIL PROTECTED]
Sent: Wednesday, November 01, 2006 9:20 PM
To: Dylan Bouterse; users@spamassassin.apache.org
Subject: Re: Relay Checker Plugin (code review please?)

You may want to download new RelayChecker.pm file...  you may have

messed

it
up previously..

 If you still have problems let me know..

- Original Message -
From: Dylan Bouterse [EMAIL PROTECTED]
To: users@spamassassin.apache.org
Sent: Wednesday, November 01, 2006 6:39 PM
Subject: RE: Relay Checker Plugin (code review please?)


 -Original Message-
 From: John D. Hardin [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, November 01, 2006 5:05 PM
 To: Dylan Bouterse
 Cc: users@spamassassin.apache.org
 Subject: RE: Relay Checker Plugin (code review please?)

 On Wed, 1 Nov 2006, Dylan Bouterse wrote:

  # headerRELAY_CHECKER   eval:relay_checker()
  # describe  RELAY_CHECKER   Check relay for DNS/Hostname

issues.

  to:
 if ($nordns) {
 
  and when I run --lint I get the following errors:
 
  /etc/mail/spamassassin/RelayChecker.pm line 44, near 27 @@

 ...how exactly did you apply the patch? From the contents of that
 error message it looks like you just inserted the patch text into

the

 source file...

 Take a look at man patch.

 (Sorry if you did do that, but that error message is really

suggestive

 of improper procedure.)


I have never used the patch command and was not aware of it. Thank you
for pointing me in the right direction. I was able to patch my
RelayChecker.cf file using the patch command and the provided patch

for

that file but I am getting errors when trying to patch the
RelayChecker.pm file.

[EMAIL PROTECTED] spamassassin]# patch -i RelayChecker.pm.patch
RelayChecker.pm
missing header for unified diff at line 3 of patch
patching file RelayChecker.pm
Hunk #3 succeeded at 102 with fuzz 1.
missing header for unified diff at line 77 of patch
can't find file to patch at input line 77
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--
|   if (! defined($name)) {
|  # the PTR record leads to a host that doesn't resolve in

DNS

|  Mail::SpamAssassin::Plugin::dbg(RelayChecker: badrdns);
|- $badrdns = 1;
|+ $badrdns = $badrdns_score;
|  }
|   else {
|  Mail::SpamAssassin::Plugin::dbg(RelayChecker: name is
$name); @@ -96,7 +123,7 @@
| # the hostname in the PTR record does resolve, but that
hostname
| # doesn't have $ip as one of its IP addresses
| Mail::SpamAssassin::Plugin::dbg(RelayChecker: baddns);
|-$baddns = 1;
|+$baddns = $baddns_score;
| }
|  else {
| ($a, $b, $c, $d) = split(/\./, $ip); # decimal octets @@
-124,7 +151,7 @@
|# in hex or decimal form ... or the entire thing in
decimal
|# probably a spambot since this is an untrusted relay
|Mail::SpamAssassin::Plugin::dbg(RelayChecker:
ipinhostname);
|-   $ipinhostname = 1;
|+   $ipinhostname = $ipinhostname_score;
|}
| if ($hostname =~
|


/(cable|catv|client|ddns|dhcp|dial-?up|dip|dsl|dynamic|ppp)\S*\.\S+\.\S+

$/
--


RelayChecker.patch
Description: Binary data

Re: Relay Checker Plugin (code review please?)

[Justin Mason]

John Rudd writes:
 Stuart Johnston wrote:
  John Rudd wrote:
  Stuart Johnston wrote:
  John Rudd wrote:
  2) This sort of replaces the other set of rules I created, that did 
  this with metarules instead of a plugin.  This made some of the 
  checks less useful.  You probably don't need to use both methods.
 
  So, what is the point of doing this as a plugin instead of using 
  existing rules?  The obvious disadvantage is the additional dns lookups.
 
  The advantages are:
 
  a) being sure that the hostname in RDNS points back to the IP address 
  you started with.  Thus detecting forgeries (which shouldn't happen 
  with _any_ legitimate service)
  
  Postfix does this for you.  It is easy enough to write an SA rule to 
  look at the Postfix headers.  I don't know about other MTAs.
 
 Sendmail does some of it, but since I didn't find detailed documentation 
 on the Trusted/Untrusted Relay pseudo-headers, I don't know if its 
 represented in there.  Nor do I know if it's on the meta-information I 
 can get from permessagestatus when I ask for the untrusted relay entries 
 (whose hash keys are, I assume, the names of the fields in the 
 trusted/untrusted relays lines)
 
 If I could get that same information without the DNS checks, I would. 
 (though, honestly, with a little more investigation, I can probably 
 eliminate ONE of my two DNS checks by looking at more of the pseudo-header).

for what it's worth: http://wiki.apache.org/spamassassin/TrustedRelays

they were woefully under-documented alright :(  now improved.

--j.

  b) just using the rules version of what I wrote, you can only check if 
  the decimal IP address, in individual segments, is in the hostname.  
  You can't check if the entire decimal IP address (one large number) is 
  in the IP address, nor can you check if the hexidecimal segments are 
  in the hostname.
 
 
  (a) requires more DNS work, yes.  (b) does not.  It just requires a 
  bit more math.
 
  
  This is just my opinion, of course, but:  I'd probably make the plugin 
  just do (b).
  
  It might be nice if SA did (a) as part of its standard checks although 
  in my experience, way too many legitimate mail servers fail on this for 
  it to be useful anyway.
 
 I have yet to have a legitimate message rejected by that check, when 
 I've been doing it in mimedefang.

Re: R: R: R: Relay Checker Plugin (code review please?)

[Andreas Pettersson]

Steven Dickenson wrote:


On Oct 31, 2006, at 6:09 AM, John Rudd wrote:

I've considered the exact opposite (adding static to the check for  
keywords).  My rules are really looking more for is this a  _client_ 
host, not is this a dynamic host.  That one check looks  for 
dynamic, but I'm not interested in exempting anyone because  
they're static.  They've still got a hostname that looks like an  
end-client, and an end-client shouldn't be connecting to other  
people's mail servers.  Any end-client that connects to someone  
else's email server should be treated like it's a spam/virus zombie



I can't agree with this.  Many small businesses in the US get just  
these kind of static connections from broadband ISPs.  Comcast, for  
example, has all of their static customers using rDNS that would fail  
your tests, and they refuse to set up a custom PTR record or delegate  
the record to someone else. 



I disagree on your disagreement. This is my opinion: If you don't have 
control over your rDNS, do NOT run any mail server, unless you relay all 
outbound mail through a server at your ISP.


Most of these static customers are  legitimate business networks 
running their own mail server, and have  neither the need nor desire 
to relay their mail through Comcast's  SMTP servers.  I think your 
general idea is very good, but you're  reaching a little too far with 
this one.



'No need nor desire', that's not really any good excuse. Use a relay or 
find your mail rejected, I'd say.


--
Andreas

Re: Relay Checker Plugin (code review please?)

[Billy Huddleston]

Attached is patch to allow scores to be done in the .cf file

--- RelayChecker.pm 2006-10-30 18:02:28.0 -0500
+++ ../RelayChecker.pm  2006-11-01 15:36:53.0 -0500
@@ -31,6 +31,12 @@
# headerRELAY_CHECKER   eval:relay_checker()
# describe  RELAY_CHECKER   Check relay for DNS/Hostname issues.

+our $base_score = 4;
+our $nordns_score = 1;
+our $badrdns_score = 1;
+our $baddns_score = 1;
+our $ipinhostname_score = 1;
+our $dynhostname_score = 1;

sub new {
   my ($class, $mailsa) = @_;
@@ -44,6 +50,27 @@
   return $self;
   }

+sub parse_config {
+my ( $self, $opts ) = @_;
+   if ( $opts-{key} eq rc_base_score ) {
+$base_score = $opts-{value};
+   }
+   elsif ( $opts-{key} eq rc_nordns_score ) {
+$nordns_score = $opts-{value};
+   }
+   elsif ( $opts-{key} eq rc_badrdns_score ) {
+$badrdns_score = $opts-{value};
+   }
+   elsif ( $opts-{key} eq rc_baddns_score ) {
+$baddns_score = $opts-{value};
+   }
+   elsif ( $opts-{key} eq rc_ipinhostname_score ) {
+$ipinhostname_score = $opts-{value};
+   }
+   elsif ( $opts-{key} eq rc_dynhostname_score ) {
+$dynhostname_score = $opts-{value};
+   }
+}

sub relay_checker {
   my ($self, $pms) = @_;
@@ -75,7 +102,7 @@
   if (! defined($hostname)) {
  # the IP address doesn't have a PTR record
  Mail::SpamAssassin::Plugin::dbg(RelayChecker: nordns);
-  $nordns = 1;
+  $nordns = $nordns_score;
  }
   else {
  ($name, $aliases, $addrtype, $length, @addrs) = 
gethostbyname($hostname);

@@ -83,7 +110,7 @@
  if (! defined($name)) {
 # the PTR record leads to a host that doesn't resolve in DNS
 Mail::SpamAssassin::Plugin::dbg(RelayChecker: badrdns);
- $badrdns = 1;
+ $badrdns = $badrdns_score;
 }
  else {
 Mail::SpamAssassin::Plugin::dbg(RelayChecker: name is $name);
@@ -96,7 +123,7 @@
# the hostname in the PTR record does resolve, but that 
hostname

# doesn't have $ip as one of its IP addresses
Mail::SpamAssassin::Plugin::dbg(RelayChecker: baddns);
-$baddns = 1;
+$baddns = $baddns_score;
}
 else {
($a, $b, $c, $d) = split(/\./, $ip); # decimal octets
@@ -124,7 +151,7 @@
   # in hex or decimal form ... or the entire thing in decimal
   # probably a spambot since this is an untrusted relay
   Mail::SpamAssassin::Plugin::dbg(RelayChecker: 
ipinhostname);

-   $ipinhostname = 1;
+   $ipinhostname = $ipinhostname_score;
   }
if ($hostname =~
  /(cable|catv|client|ddns|dhcp|dial-?up|dip|dsl|dynamic|ppp)\S*\.\S+\.\S+$/
@@ -136,7 +163,7 @@
   # hostname contains words that look dynamic
   # probably a spambot since this is an untrusted relay
   Mail::SpamAssassin::Plugin::dbg(RelayChecker: 
dynhostname);

-   $dynhostname = 1;
+   $dynhostname = $dynhostname_score;
   }

} # found ip addr
@@ -145,7 +172,7 @@

   $score = $nordns + $badrdns + $baddns + $ipinhostname + $dynhostname;
   if ($score) {
-  $score += 4;
+  $score += $base_score;
  my $description = $pms-{conf}-{description}-{RELAY_CHECKER};

  if ($nordns) {


--- RelayChecker.cf 2006-10-30 18:02:28.0 -0500
+++ ../RelayChecker.cf  2006-11-01 15:38:30.0 -0500
@@ -7,4 +7,9 @@
loadplugin  RelayCheckerRelayChecker.pm
header  RELAY_CHECKER   eval:relay_checker()
describeRELAY_CHECKER   Check relay for DNS/Hostname issues
-
+rc_base_score  1.4
+rc_nordns_score1
+rc_badrdns_score   1
+rc_baddns_score1
+rc_ipinhostname_score  1
+rc_dynhostname_score   1







- Original Message - 
From: Andreas Pettersson [EMAIL PROTECTED]

To: Steven Dickenson [EMAIL PROTECTED]
Cc: John Rudd [EMAIL PROTECTED]; Giampaolo Tomassoni 
[EMAIL PROTECTED]; users@spamassassin.apache.org

Sent: Wednesday, November 01, 2006 12:11 PM
Subject: Re: R: R: R: Relay Checker Plugin (code review please?)



Steven Dickenson wrote:


On Oct 31, 2006, at 6:09 AM, John Rudd wrote:

I've considered the exact opposite (adding static to the check for 
keywords).  My rules are really looking more for is this a  _client_ 
host, not is this a dynamic host.  That one check looks  for 
dynamic, but I'm not interested in exempting anyone because  they're 
static.  They've still got a hostname that looks like an  end-client, 
and an end-client shouldn't be connecting to other  people's mail 
servers.  Any end-client that connects to someone  else's email server 
should be treated like it's a spam/virus zombie



I can't agree with this.  Many small businesses in the US get just  these 
kind of static connections from broadband ISPs.  Comcast, for  example, 
has all of their static customers using rDNS that would fail  your tests, 
and they refuse to set up a custom PTR

RE: Relay Checker Plugin (code review please?)

[Dylan Bouterse]

 -Original Message-
 From: John D. Hardin [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, November 01, 2006 5:05 PM
 To: Dylan Bouterse
 Cc: users@spamassassin.apache.org
 Subject: RE: Relay Checker Plugin (code review please?)
 
 On Wed, 1 Nov 2006, Dylan Bouterse wrote:
 
  # headerRELAY_CHECKER   eval:relay_checker()
  # describe  RELAY_CHECKER   Check relay for DNS/Hostname issues.
  to:
 if ($nordns) {
 
  and when I run --lint I get the following errors:
 
  /etc/mail/spamassassin/RelayChecker.pm line 44, near 27 @@
 
 ...how exactly did you apply the patch? From the contents of that
 error message it looks like you just inserted the patch text into the
 source file...
 
 Take a look at man patch.
 
 (Sorry if you did do that, but that error message is really suggestive
 of improper procedure.)
 

I have never used the patch command and was not aware of it. Thank you
for pointing me in the right direction. I was able to patch my
RelayChecker.cf file using the patch command and the provided patch for
that file but I am getting errors when trying to patch the
RelayChecker.pm file.

[EMAIL PROTECTED] spamassassin]# patch -i RelayChecker.pm.patch
RelayChecker.pm
missing header for unified diff at line 3 of patch
patching file RelayChecker.pm
Hunk #3 succeeded at 102 with fuzz 1.
missing header for unified diff at line 77 of patch
can't find file to patch at input line 77
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--
|   if (! defined($name)) {
|  # the PTR record leads to a host that doesn't resolve in DNS
|  Mail::SpamAssassin::Plugin::dbg(RelayChecker: badrdns);
|- $badrdns = 1;
|+ $badrdns = $badrdns_score;
|  }
|   else {
|  Mail::SpamAssassin::Plugin::dbg(RelayChecker: name is
$name); @@ -96,7 +123,7 @@
| # the hostname in the PTR record does resolve, but that
hostname
| # doesn't have $ip as one of its IP addresses
| Mail::SpamAssassin::Plugin::dbg(RelayChecker: baddns);
|-$baddns = 1;
|+$baddns = $baddns_score;
| }
|  else {
| ($a, $b, $c, $d) = split(/\./, $ip); # decimal octets @@
-124,7 +151,7 @@
|# in hex or decimal form ... or the entire thing in
decimal
|# probably a spambot since this is an untrusted relay
|Mail::SpamAssassin::Plugin::dbg(RelayChecker:
ipinhostname);
|-   $ipinhostname = 1;
|+   $ipinhostname = $ipinhostname_score;
|}
| if ($hostname =~
|
/(cable|catv|client|ddns|dhcp|dial-?up|dip|dsl|dynamic|ppp)\S*\.\S+\.\S+
$/
--

Re: Relay Checker Plugin (code review please?)

[Billy Huddleston]

You may want to download new RelayChecker.pm file...  you may have messed it 
up previously..


If you still have problems let me know..

- Original Message - 
From: Dylan Bouterse [EMAIL PROTECTED]

To: users@spamassassin.apache.org
Sent: Wednesday, November 01, 2006 6:39 PM
Subject: RE: Relay Checker Plugin (code review please?)



-Original Message-
From: John D. Hardin [mailto:[EMAIL PROTECTED]
Sent: Wednesday, November 01, 2006 5:05 PM
To: Dylan Bouterse
Cc: users@spamassassin.apache.org
Subject: RE: Relay Checker Plugin (code review please?)

On Wed, 1 Nov 2006, Dylan Bouterse wrote:

 # headerRELAY_CHECKER   eval:relay_checker()
 # describe  RELAY_CHECKER   Check relay for DNS/Hostname issues.
 to:
if ($nordns) {

 and when I run --lint I get the following errors:

 /etc/mail/spamassassin/RelayChecker.pm line 44, near 27 @@

...how exactly did you apply the patch? From the contents of that
error message it looks like you just inserted the patch text into the
source file...

Take a look at man patch.

(Sorry if you did do that, but that error message is really suggestive
of improper procedure.)



I have never used the patch command and was not aware of it. Thank you
for pointing me in the right direction. I was able to patch my
RelayChecker.cf file using the patch command and the provided patch for
that file but I am getting errors when trying to patch the
RelayChecker.pm file.

[EMAIL PROTECTED] spamassassin]# patch -i RelayChecker.pm.patch
RelayChecker.pm
missing header for unified diff at line 3 of patch
patching file RelayChecker.pm
Hunk #3 succeeded at 102 with fuzz 1.
missing header for unified diff at line 77 of patch
can't find file to patch at input line 77
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--
|   if (! defined($name)) {
|  # the PTR record leads to a host that doesn't resolve in DNS
|  Mail::SpamAssassin::Plugin::dbg(RelayChecker: badrdns);
|- $badrdns = 1;
|+ $badrdns = $badrdns_score;
|  }
|   else {
|  Mail::SpamAssassin::Plugin::dbg(RelayChecker: name is
$name); @@ -96,7 +123,7 @@
| # the hostname in the PTR record does resolve, but that
hostname
| # doesn't have $ip as one of its IP addresses
| Mail::SpamAssassin::Plugin::dbg(RelayChecker: baddns);
|-$baddns = 1;
|+$baddns = $baddns_score;
| }
|  else {
| ($a, $b, $c, $d) = split(/\./, $ip); # decimal octets @@
-124,7 +151,7 @@
|# in hex or decimal form ... or the entire thing in
decimal
|# probably a spambot since this is an untrusted relay
|Mail::SpamAssassin::Plugin::dbg(RelayChecker:
ipinhostname);
|-   $ipinhostname = 1;
|+   $ipinhostname = $ipinhostname_score;
|}
| if ($hostname =~
|
/(cable|catv|client|ddns|dhcp|dial-?up|dip|dsl|dynamic|ppp)\S*\.\S+\.\S+
$/
--

RE: Relay Checker Plugin (code review please?)

[Dylan Bouterse]

I did a couple of times. :(

 -Original Message-
 From: Billy Huddleston [mailto:[EMAIL PROTECTED]
 Sent: Wednesday, November 01, 2006 9:20 PM
 To: Dylan Bouterse; users@spamassassin.apache.org
 Subject: Re: Relay Checker Plugin (code review please?)
 
 You may want to download new RelayChecker.pm file...  you may have
messed
 it
 up previously..
 
  If you still have problems let me know..
 
 - Original Message -
 From: Dylan Bouterse [EMAIL PROTECTED]
 To: users@spamassassin.apache.org
 Sent: Wednesday, November 01, 2006 6:39 PM
 Subject: RE: Relay Checker Plugin (code review please?)
 
 
  -Original Message-
  From: John D. Hardin [mailto:[EMAIL PROTECTED]
  Sent: Wednesday, November 01, 2006 5:05 PM
  To: Dylan Bouterse
  Cc: users@spamassassin.apache.org
  Subject: RE: Relay Checker Plugin (code review please?)
 
  On Wed, 1 Nov 2006, Dylan Bouterse wrote:
 
   # headerRELAY_CHECKER   eval:relay_checker()
   # describe  RELAY_CHECKER   Check relay for DNS/Hostname
issues.
   to:
  if ($nordns) {
  
   and when I run --lint I get the following errors:
  
   /etc/mail/spamassassin/RelayChecker.pm line 44, near 27 @@
 
  ...how exactly did you apply the patch? From the contents of that
  error message it looks like you just inserted the patch text into
the
  source file...
 
  Take a look at man patch.
 
  (Sorry if you did do that, but that error message is really
suggestive
  of improper procedure.)
 
 
 I have never used the patch command and was not aware of it. Thank you
 for pointing me in the right direction. I was able to patch my
 RelayChecker.cf file using the patch command and the provided patch
for
 that file but I am getting errors when trying to patch the
 RelayChecker.pm file.
 
 [EMAIL PROTECTED] spamassassin]# patch -i RelayChecker.pm.patch
 RelayChecker.pm
 missing header for unified diff at line 3 of patch
 patching file RelayChecker.pm
 Hunk #3 succeeded at 102 with fuzz 1.
 missing header for unified diff at line 77 of patch
 can't find file to patch at input line 77
 Perhaps you should have used the -p or --strip option?
 The text leading up to this was:
 --
 |   if (! defined($name)) {
 |  # the PTR record leads to a host that doesn't resolve in
DNS
 |  Mail::SpamAssassin::Plugin::dbg(RelayChecker: badrdns);
 |- $badrdns = 1;
 |+ $badrdns = $badrdns_score;
 |  }
 |   else {
 |  Mail::SpamAssassin::Plugin::dbg(RelayChecker: name is
 $name); @@ -96,7 +123,7 @@
 | # the hostname in the PTR record does resolve, but that
 hostname
 | # doesn't have $ip as one of its IP addresses
 | Mail::SpamAssassin::Plugin::dbg(RelayChecker: baddns);
 |-$baddns = 1;
 |+$baddns = $baddns_score;
 | }
 |  else {
 | ($a, $b, $c, $d) = split(/\./, $ip); # decimal octets @@
 -124,7 +151,7 @@
 |# in hex or decimal form ... or the entire thing in
 decimal
 |# probably a spambot since this is an untrusted relay
 |Mail::SpamAssassin::Plugin::dbg(RelayChecker:
 ipinhostname);
 |-   $ipinhostname = 1;
 |+   $ipinhostname = $ipinhostname_score;
 |}
 | if ($hostname =~
 |

/(cable|catv|client|ddns|dhcp|dial-?up|dip|dsl|dynamic|ppp)\S*\.\S+\.\S+
 $/
 --

R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

 So, if people could take a look at it, test it, see if it does what it 
 advertises, and see if it's as accurate as my experience indicates, I 
 would appreciate getting feedback.  If it pans out, I'll see about 
 putting it in a tar ball, and submitting it to the wiki's list of plugins.

if ( ($hostname =~ /(\S?0*($a|$b|$c|$d|$e|$f|$g|$h|$i)){2,4}/) ||
 ($hostname =~ /$e/) ) {
   # hostname contains two or more octets of its own IP addr
   # in hex or decimal form ... or the entire thing in decimal
   # probably a spambot since this is an untrusted relay
   Mail::SpamAssassin::Plugin::dbg(RelayChecker: ipinhostname);
   $ipinhostname = 1;
   }

Wow, how rude this is! Almost all customers of my ISP (Telecom Italia) would be 
banned from the e-mail world...

Telecom Italia is used to put RDNSes with something like this:

 host1-84-static.48-88-b.business.telecomitalia.it.

Cheers,

---
Giampaolo Tomassoni - IT Consultant
Piazza VIII Aprile 1948, 4
I-53044 Chiusi (SI) - Italy
Ph: +39-0578-21100

MAI inviare una e-mail a:
NEVER send an e-mail to:
 [EMAIL PROTECTED]

 
 
 John
 

Re: R: Relay Checker Plugin (code review please?)

[Alain Wolf]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 31.10.2006 09:13, * Giampaolo Tomassoni wrote:
 So, if people could take a look at it, test it, see if it does what it 
 advertises, and see if it's as accurate as my experience indicates, I 
 would appreciate getting feedback.  If it pans out, I'll see about 
 putting it in a tar ball, and submitting it to the wiki's list of plugins.
 
 if ( ($hostname =~ /(\S?0*($a|$b|$c|$d|$e|$f|$g|$h|$i)){2,4}/) ||
  ($hostname =~ /$e/) ) {
# hostname contains two or more octets of its own IP addr
# in hex or decimal form ... or the entire thing in decimal
# probably a spambot since this is an untrusted relay
Mail::SpamAssassin::Plugin::dbg(RelayChecker: ipinhostname);
$ipinhostname = 1;
}
 
 Wow, how rude this is! Almost all customers of my ISP (Telecom Italia) would 
 be banned from the e-mail world...
 
 Telecom Italia is used to put RDNSes with something like this:
 
  host1-84-static.48-88-b.business.telecomitalia.it.
 
 Cheers,
 
 ---
 Giampaolo Tomassoni - IT Consultant
 Piazza VIII Aprile 1948, 4
 I-53044 Chiusi (SI) - Italy
 Ph: +39-0578-21100
 
 MAI inviare una e-mail a:
 NEVER send an e-mail to:
  [EMAIL PROTECTED]
 

 John

 
 

Same here in Switzerland, at least one of the main national ISPs calls
his clients nn-nn-nn-nn.static.cablecom.ch

But we had already rejections and spam-tags from many places even before
that plugin came out. But they give you a reverse DNS entry of your own
hostname if you ask for.




-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFRwgkV5MZZmyxvGgRAv5wAKDTycC4mesnutBGmaCdaJR6nSl01gCgx71a
wzXKhjS1sbFk8LCX1oEyfzI=
=0GOX
-END PGP SIGNATURE-

R: R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

 Same here in Switzerland, at least one of the main national ISPs calls
 his clients nn-nn-nn-nn.static.cablecom.ch
 
 But we had already rejections and spam-tags from many places even before
 that plugin came out. But they give you a reverse DNS entry of your own
 hostname if you ask for.

Well, you know, swiss is well known to be exact.

Here in Italy it is a bit more difficult to get a RDNS changed by Telecom 
Italia: FWIK, they really don't care about RDNS and have no defined policies 
about it.

---
Giampaolo Tomassoni - IT Consultant
Piazza VIII Aprile 1948, 4
I-53044 Chiusi (SI) - Italy
Ph: +39-0578-21100

MAI inviare una e-mail a:
NEVER send an e-mail to:
 [EMAIL PROTECTED]


 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.5 (MingW32)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
 iD8DBQFFRwgkV5MZZmyxvGgRAv5wAKDTycC4mesnutBGmaCdaJR6nSl01gCgx71a
 wzXKhjS1sbFk8LCX1oEyfzI=
 =0GOX
 -END PGP SIGNATURE-
 

R: R: R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

 On 31.10.2006 09:32, * Giampaolo Tomassoni wrote:
  Same here in Switzerland, at least one of the main national ISPs calls
  his clients nn-nn-nn-nn.static.cablecom.ch
 
  But we had already rejections and spam-tags from many places 
 even before
  that plugin came out. But they give you a reverse DNS entry of your own
  hostname if you ask for.
  
  Well, you know, swiss is well known to be exact.
  
  Here in Italy it is a bit more difficult to get a RDNS changed 
 by Telecom Italia: FWIK, they really don't care about RDNS and 
 have no defined policies about it.
  
  
 
 A few months ago the said addresses were called
 nn-nn-nn-nn.webcom.cablecom.ch until that day when SORBS just put all
 these netblocks in its RBL as dynamic. And they refused to take it out
 until the ISP changed the names to todays nn-nn-nn-nn.static.cablecom.ch
 
 So it looks to me that this plugin should exclude hosts which have
 *static*, *sta* or *fixed* in their DNS names.

I agree with this.


 SORBS uses the following Internet Draft for determining whether networks
 are statically or dynamically by rDNS:
 http://tools.ietf.org/wg/dnsop/draft-msullivan-dnsop-generic-namin
 g-schemes-00.txt

Right. Also, SORBS goes a bit (too?) further by including the pool word in 
RDNS as a dynamic address indicator. This sounds not that correct to me.

(Again) Telecom Italia uses it to mark address pools on statically-assigned 
chunks:

 host1-231.pool8175.interbusiness.it.

This means the host 231.1 in the 81.75 address pool and, believe me, has 
nothing to do with dynamic addresses: that's statically assigned (uses CLIP, 
too...).


 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.5 (MingW32)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
 iD8DBQFFRxCLV5MZZmyxvGgRAkiZAKDX361SHB3MOeQaMtBmbPLHiccJBACePirl
 CIkcQgKV3DkAWRI8UDfdmGQ=
 =QKJl
 -END PGP SIGNATURE-
 

Re: R: Relay Checker Plugin (code review please?)

[John Rudd]

Giampaolo Tomassoni wrote:
So, if people could take a look at it, test it, see if it does what it 
advertises, and see if it's as accurate as my experience indicates, I 
would appreciate getting feedback.  If it pans out, I'll see about 
putting it in a tar ball, and submitting it to the wiki's list of plugins.


if ( ($hostname =~ /(\S?0*($a|$b|$c|$d|$e|$f|$g|$h|$i)){2,4}/) ||
 ($hostname =~ /$e/) ) {
   # hostname contains two or more octets of its own IP addr
   # in hex or decimal form ... or the entire thing in decimal
   # probably a spambot since this is an untrusted relay
   Mail::SpamAssassin::Plugin::dbg(RelayChecker: ipinhostname);
   $ipinhostname = 1;
   }

Wow, how rude this is! Almost all customers of my ISP (Telecom Italia) would be 
banned from the e-mail world...

Telecom Italia is used to put RDNSes with something like this:

 host1-84-static.48-88-b.business.telecomitalia.it.



They would not be banned from the e-mail world.

Instead, they would:

a) be heavily encouraged to get a custom RDNS record, OR
b) be heavily encouraged to send outgoing email through their ISP*, OR
c) be heavily encouraged to use a hosted email service that has a custom 
RDNS record instead of a client-looking RDNS record, OR

d) accept that their email is going to be quarantined (not banned).


(*  which they should do -- I'm not their email server, so unless they 
can make themselves look like a server, instead of a client, they have 
no business connecting directly to my email server; they should connect 
to their own email server, which should have a custom RDNS record, and 
then have that machine connect to my email server)


If they can't do (a) because their ISP doesn't offer that, then they'd 
be encouraged to switch to an ISP that does offer custom RNDS records 
... or do (b) or (c).


I'm personally comfortable with insisting that the people who want to 
connect to my email servers conform to those options.  It's certainly a 
nicer set of options than having (d) be: accept that their email wont be 
accepted at all (which is what I've done in the past).

Re: R: R: R: Relay Checker Plugin (code review please?)

[John Rudd]

Giampaolo Tomassoni wrote:

On 31.10.2006 09:32, * Giampaolo Tomassoni wrote:

Same here in Switzerland, at least one of the main national ISPs calls
his clients nn-nn-nn-nn.static.cablecom.ch

But we had already rejections and spam-tags from many places 

even before

that plugin came out. But they give you a reverse DNS entry of your own
hostname if you ask for.

Well, you know, swiss is well known to be exact.

Here in Italy it is a bit more difficult to get a RDNS changed 
by Telecom Italia: FWIK, they really don't care about RDNS and 
have no defined policies about it.



A few months ago the said addresses were called
nn-nn-nn-nn.webcom.cablecom.ch until that day when SORBS just put all
these netblocks in its RBL as dynamic. And they refused to take it out
until the ISP changed the names to todays nn-nn-nn-nn.static.cablecom.ch

So it looks to me that this plugin should exclude hosts which have
*static*, *sta* or *fixed* in their DNS names.


I agree with this.


I've considered the exact opposite (adding static to the check for 
keywords).  My rules are really looking more for is this a _client_ 
host, not is this a dynamic host.  That one check looks for 
dynamic, but I'm not interested in exempting anyone because they're 
static.  They've still got a hostname that looks like an end-client, 
and an end-client shouldn't be connecting to other people's mail 
servers.  Any end-client that connects to someone else's email server 
should be treated like it's a spam/virus zombie.




SORBS uses the following Internet Draft for determining whether networks
are statically or dynamically by rDNS:
http://tools.ietf.org/wg/dnsop/draft-msullivan-dnsop-generic-namin
g-schemes-00.txt


Right. Also, SORBS goes a bit (too?) further by including the pool word in 
RDNS as a dynamic address indicator. This sounds not that correct to me.



I've also thought about adding pool to my list of keywords ... I just 
thought it might be a little too generic.

Re: R: R: Relay Checker Plugin (code review please?)

[John Rudd]

Alain Wolf wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 31.10.2006 09:32, * Giampaolo Tomassoni wrote:

Same here in Switzerland, at least one of the main national ISPs calls
his clients nn-nn-nn-nn.static.cablecom.ch

But we had already rejections and spam-tags from many places even before
that plugin came out. But they give you a reverse DNS entry of your own
hostname if you ask for.

Well, you know, swiss is well known to be exact.

Here in Italy it is a bit more difficult to get a RDNS changed by Telecom 
Italia: FWIK, they really don't care about RDNS and have no defined policies 
about it.




A few months ago the said addresses were called
nn-nn-nn-nn.webcom.cablecom.ch until that day when SORBS just put all
these netblocks in its RBL as dynamic. And they refused to take it out
until the ISP changed the names to todays nn-nn-nn-nn.static.cablecom.ch

So it looks to me that this plugin should exclude hosts which have
*static*, *sta* or *fixed* in their DNS names.

SORBS uses the following Internet Draft for determining whether networks
are statically or dynamically by rDNS:
http://tools.ietf.org/wg/dnsop/draft-msullivan-dnsop-generic-naming-schemes-00.txt




It should only exempt static hosts if the larger rule is targeting 
dynamic hosts.  That one regular expression is after dynamic hosts ... 
but the larger rule is after clients, not dynamic hosts.  Therefore, 
exempting static or fixed hostnames doesn't fit.

R: R: R: R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

 ...omissis...

 I've considered the exact opposite (adding static to the check for 
 keywords).  My rules are really looking more for is this a _client_ 
 host, not is this a dynamic host.  That one check looks for 
 dynamic, but I'm not interested in exempting anyone because they're 
 static.  They've still got a hostname that looks like an end-client, 
 and an end-client shouldn't be connecting to other people's mail 
 servers.  Any end-client that connects to someone else's email server 
 should be treated like it's a spam/virus zombie.

I'm not comfortable with this: the border between an end-client and a server is 
really unclean. Also, what about and end-client server? :)

I don't understand the push toward using the ISP's mail server to send mail. I 
guess that an end-client may legitimally run its own mail server without 
relaing its outgoing mail to its internet provider.

I can, however, well understand the need for a legitimate mx to be tied to a 
static address. That make sense for identification purposes.

What's wrong with small businesses running their own mx? Just guessing: isn't 
that the blame about this originates from large ISPs that just want to tight 
their customers?

---
Giampaolo Tomassoni - IT Consultant
Piazza VIII Aprile 1948, 4
I-53044 Chiusi (SI) - Italy
Ph: +39-0578-21100

MAI inviare una e-mail a:
NEVER send an e-mail to:
 [EMAIL PROTECTED]



 
 
  SORBS uses the following Internet Draft for determining 
 whether networks
  are statically or dynamically by rDNS:
  http://tools.ietf.org/wg/dnsop/draft-msullivan-dnsop-generic-namin
  g-schemes-00.txt
  
  Right. Also, SORBS goes a bit (too?) further by including the 
 pool word in RDNS as a dynamic address indicator. This sounds 
 not that correct to me.
  
 
 I've also thought about adding pool to my list of keywords ... I just 
 thought it might be a little too generic.
 

R: R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

  I would prefer not to have to deal with a single, computed 
 RELAY_CHECKER score, but with many different ones for each of the 
 triggered cases. This way it would be easier to tune scores from 
 this plugin.
  
  To me, your plugin could trigger the following tags:
  
  RELAY_CHECKER (at least one rule had been triggered. According 
 to your code would score 4 by default);
  RC_NORDNS (scores 1);
  RC_BADRDNS  (scores 1);
  RC_BADDNS   (scores 1);
  RC_IPINHOSTNAME (scores 1);
  RC_DYNHOSTNAME  (scores 1);
  
 
 I was actually thinking of something slightly different.
 
 One static score that can be adjusted in the cf file.  Say, 6 (this 
 makes more sense than the current situation of sometimes you get 5, 
 sometimes you get 6, in my opinion).
 
 Then a bunch of individual scores (like you suggest) that are 
 dynamically scored (the way the plugin records its current score, giving 
 each of those hits as 0 or .01).
 
 This would give a score range of 6.01 to 6.05.  The basic idea is if 
 you get hit by this plugin at all, you're going to get a 6, but the .01 
 scores will show up in a detailed report header, letting you know which 
 specific characteristics were triggered.
 
 
 When someone wants to run tests, they'd just set the static score from 6 
 to .01 (yielding an overall score from .01 to .05).

My intention was to use this plugin for some checks but not for others.

I would assign 0 score to RELAY_CHECKER, RC_BADDNS and RC_IPINHOSTNAME, then 
the score I like to, say, RC_DYNHOSTNAME, RC_NORDNS and RC_BADRDNS (maybe a 1 
to 2 score).

I would like to use this plugin to give hints to my SA, not to definitely stop 
a source. :)


 The other two things I'm looking at changing are:
 a) having a relaycheck_exempt cf configuration,
 b) looking at the auth part of the untrusted relay data.
 
 The result would be that instead of looking at the first untrusted 
 relay, it would skip past untrusted relays that were in the 
 relaycheck_exempt list.  Then, if the untrusted relay it's left with had 
 used authentication, the rule wouldn't trigger.

Fine.

---
Giampaolo Tomassoni - IT Consultant
Piazza VIII Aprile 1948, 4
I-53044 Chiusi (SI) - Italy
Ph: +39-0578-21100

MAI inviare una e-mail a:
NEVER send an e-mail to:
 [EMAIL PROTECTED]

Re: R: Relay Checker Plugin (code review please?)

[John Rudd]

Massimiliano Hofer wrote:
We have 
rather successfull anti-spam legislation and, except for botnets, really 
little spam originates here.




Right ... but it's those botnets that this plugin is trying to catch.


And, while I may be a little unyielding wrt to people whose ISPs are 
like Telecom Italia, I'm not unsympathetic.  I think, in this case, if 
Italy did get mass quarantined by the rest of the world, it might cause 
enough of an uproar to force Telecom Italia to change its practices and 
allow custom RDNS.  That wont make your life any easier in the meantime, 
though.  I understand that ... but I honestly think it's the right stand 
to take from my side of each SMTP transaction.


I suppose the rate at which people may or may not adopt this plugin when 
it's finished will tell us how many people agree with my stance.

Re: R: Relay Checker Plugin (code review please?)

[Jonas Eckerman]

Giampaolo Tomassoni wrote:


RELAY_CHECKER (at least one rule had been triggered. According to your code 
would score 4 by default);
RC_NORDNS (scores 1);
RC_BADRDNS  (scores 1);
RC_BADDNS   (scores 1);
RC_IPINHOSTNAME (scores 1);
RC_DYNHOSTNAME  (scores 1);


Agreed. This way the plugin could also add some rules for ham.

I'm doing something similar myself in MIMEDefang. I've got a number of checks. 
My resulting rules (applyed after the SA checks) are:

IP_FQDN_0 - IP_FQDN_5
USER_FQDN_0 - USER_FQDN_3
MAIL_FQDN_0 - MAIL_FQDN_3
NO_FQDN_0 - NO_FQDN_1

and I can then use meta rules for the scoring based on those results.
I don't know if such fine grained rules are really needed for this.

The MAIL_FQDN_* rules are ham-signs from this check:

sub check_mail_fqdn {
my $fqdn = shift;
my $xxx = '(mail|relay|smtp|out)';
return 3 if ($fqdn =~ /^(|.*[._-])$xxx\d{0,5}(|[._-].*)$/i);
return 2 if ($fqdn =~ /^(|.*[._-])$xxx[-._]?$xxx\d{0,5}(|[._-].*)$/i);
return 1 if ($fqdn =~ /(mail|smtp|relay)/i);
return 0;
}

That should be changed to include static in $xxx.

Just for the sake of comparison, below are the other checks as well:

---8---
sub check_ip_parts {
my $x = shift;
return 0 if ($x  @_ != 4);
my $ic = 0;
my $hc = 0;
foreach my $p (@_) {
unless ($x) {
my @pp = split(/-/,$p);
return 3 if (check_ip_parts(1,@pp));
@pp = split(/_/,$p);
return 3 if (check_ip_parts(1,@pp));
}
my $i = ($p =~ /^\d{1,3}$/  $p = 0  $p = 255);
my $h = 0;
if ($p =~ /^[0-9A-Fa-f]{1,2}$/) {
my $i = hex $p;
$h = ($i = 0  $i = 255);
}
$ic ++ if ($i);
$hc ++ if ($h);
return 2 if ($ic == 4);
return 1 if ($hc == 4);
}
return 0;
}

sub check_ip_fqdn {
my $fqdn = shift;
my $ip = shift;
return 0 if ($fqdn =~ /^\[$ip\]$/);
if ($ip =~ /^\d+\.\d+\.\d+\.\d+$/) {
my $rip = join('.',reverse split(/\./,$ip));
$ip =~ 
s/(\d+)/sprintf('(%1$u|%1$x|%1$02u|%1$02x|%1$03u)',$1)/ge;
$rip =~ 
s/(\d+)/sprintf('(%1$u|%1$x|%1$02u|%1$02x|%1$03u)',$1)/ge;
$ip =~ s/\./[-._]/g;
$rip =~ s/\./[-._]/g;
return 5 if ($fqdn =~ /(|.*\.)$ip\./i);
return 5 if ($fqdn =~ /(|.*\.)$rip\./i);
$ip =~ s/\[-\._\]//g;
$rip =~ s/\[-\._\]//g;
return 4 if ($fqdn =~ /(|.*\.)$ip\./i);
return 4 if ($fqdn =~ /(|.*\.)$rip\./i);
}
return check_ip_parts(0,split(/\./,$fqdn));
}

sub check_user_fqdn {
my $fqdn = shift;
return 3 if ($fqdn =~ 
/^(|.*[._-])(a?dsl|cable|dial[-._]?up|dynamic|dynamicip|customer|dhcp)(|[._-].*)$/i);
return 2 if ($fqdn =~ /^(|.*[._-])(cust|kund)(|[._-].*)$/i);
return 1 if ($fqdn =~ /^(|.*[._-])(a?dsl[a-z]|cable)\d*(|[._-].*)$/i);
return 0;
}

sub check_mail_fqdn {
my $fqdn = shift;
my $xxx = '(mail|relay|smtp|out)';
return 3 if ($fqdn =~ /^(|.*[._-])$xxx\d{0,5}(|[._-].*)$/i);
return 2 if ($fqdn =~ /^(|.*[._-])$xxx[-._]?$xxx\d{0,5}(|[._-].*)$/i);
return 1 if ($fqdn =~ /(mail|smtp|relay)/i);
return 0;
}
---8---

Regards
/Jonas

--
Jonas Eckerman, FSDB  Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/

R: R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

 Massimiliano Hofer wrote:
  We have 
  rather successfull anti-spam legislation and, except for 
 botnets, really 
  little spam originates here.
  
 
 Right ... but it's those botnets that this plugin is trying to catch.

I use greylisting for this, and it works great to me. Also, it simply 
challenges the peer about some Rfc 2821 compliance (a 4xx error is a temporary 
one and every good 2821-compliant server MUST retry).


 And, while I may be a little unyielding wrt to people whose ISPs are 
 like Telecom Italia, I'm not unsympathetic.  I think, in this case, if 
 Italy did get mass quarantined by the rest of the world, it might cause 
 enough of an uproar to force Telecom Italia to change its practices and 
 allow custom RDNS.  That wont make your life any easier in the meantime, 
 though.  I understand that ... but I honestly think it's the right stand 
 to take from my side of each SMTP transaction.

The problem is not only Telecom Italia (who, besides, may even care nothing 
about their customers' mail being dropped: it's basicly a monopoly).

I see also a theoretical one. Internet is meant to be a medium with much more 
freedom than other ones.

Basicly, the main idea behind internet is that you get a static IP and you do 
whatever (legal) thing you like with it, without having to further rely on your 
connectivity provider for this.

This include even run a legitimate mx. There is no RFC stating you need to 
relay your mail to your ISP if you're too small. And it wouldn't make sense as 
long as even RFCs (i.e.: the interoperability standard) are available to 
everybody for free.

This is a concept which is far away from other media. Try to get ITU-T or ANSI 
standards for free: while you have to be a big company if you want to run your 
own telephone system, it isn't needed to run your own mx.

Of course, this doesn't mean that the destinator of an e-mail has to accept 
each and every e-mails: he/she too has the freedom to accept or discard it. But 
I wouldn't like to be discriminated just because of my company's size: this is 
well out of the Internet idea.

By strictly enforcing DNS/RDNS ruling you basicly discriminate small companies 
(the ones that can't afford buying a /24 net from Ripe or Arin and run their 
own RDNS) from the big ones (the ones for which a /24 would even be ridiculus). 
You are not going to create troubles to Telecom Italia this way, you are going 
to help them to stay in their big business: their customers will be enforced to 
use Telecom's servers to relay mail, which means to have to adjust to their 
off-service schedules and maybe even e-mail policies. Actually it doesn't 
happen, but what if Telecom wakes up in a morning with the idea that its 
customers have to pay a fee for each domain for which they relay mail through 
its servers?

This is why I think that your plugin is a useful mean to give hints to SA, but 
I would like to definitely lower its scores.


 I suppose the rate at which people may or may not adopt this plugin when 
 it's finished will tell us how many people agree with my stance.

Not quite, if they lower the scores... :)


---
Giampaolo Tomassoni - IT Consultant
Piazza VIII Aprile 1948, 4
I-53044 Chiusi (SI) - Italy
Ph: +39-0578-21100

MAI inviare una e-mail a:
NEVER send an e-mail to:
 [EMAIL PROTECTED]

Re: Relay Checker Plugin (code review please?)

[Rick Macdougall]

John Rudd wrote:

Rick Macdougall wrote:

John Rudd wrote:




Hi,

Right off the bat I've disabled it.  It, of course, hits on all mail 
my local users send.  That's not really acceptable in an ISP situation 
so I've turned it off until tomorrow when I have the time to look at 
the code and see if I can disable the check for specific IP's or host 
names.


I can say it was hitting on a lot of spam that was passing through as 
clean before, so there is quite a bit of merit to the idea.  It would 
just need the ability to ignore local clients.




Are those users on your trusted network?  It should only be looking at 
your first untrusted relay.


Though, if they're authenticated, I wouldn't mind trying to figure out 
how to extract that from the information, and exempt those.


I could easily add a list of exemptions though.



Hi,

No, they aren't in my trusted networks because I don't trust them.

The reasoning behind the scanning is to pro-actively catch infected 
users spewing spam before much damage is done.  We run a script every 5 
minutes to check for local IP's that are sending spam and if we get a 
pre-defined number of matches it sends us an email.


I may try it later today on one of our external facing MX servers and 
see how it fairs there. (After coffee and fully waking up).


Regards,

Rick

RE: R: R: R: Relay Checker Plugin (code review please?)

[Coffey, Neal]

John Rudd wrote:
 I've considered the exact opposite (adding static to the check for
 keywords). [...]  They've still got a hostname that looks like an
 end-client, and an end-client shouldn't be connecting to other
 people's mail servers.  Any end-client that connects to someone
 else's email server should be treated like it's a spam/virus zombie.

Except that addresses from the Static pools are typically given to
customers of the small business packages, specifically for the purpose
of running their own servers. (For cable operators in the US, that is
basically the only difference between the residental and the small
business packages.)

So what you're really saying is, They've got a hostname that looks like
a small business, and a small business shouldn't be connecting to other
people's mail servers. Now, some of the ISPs do let residential
customers pay extra for a static address.  But I'm willing to wager that
anyone who's paying extra for a static IP is going to be smarter than
the average bear, and not let themselves get zombified.

I like what you've got so far (though I haven't put it on my own server
yet...looking for more feedback from others first), but I disagree with
adding static to the keywords.

Re: R: R: R: Relay Checker Plugin (code review please?)

[John Rudd]

Steven Dickenson wrote:

On Oct 31, 2006, at 6:09 AM, John Rudd wrote:

I've considered the exact opposite (adding static to the check for 
keywords).  My rules are really looking more for is this a _client_ 
host, not is this a dynamic host.  That one check looks for 
dynamic, but I'm not interested in exempting anyone because they're 
static.  They've still got a hostname that looks like an end-client, 
and an end-client shouldn't be connecting to other people's mail 
servers.  Any end-client that connects to someone else's email server 
should be treated like it's a spam/virus zombie


I can't agree with this.  Many small businesses in the US get just these 
kind of static connections from broadband ISPs.  Comcast, for example, 
has all of their static customers using rDNS that would fail your tests, 
and they refuse to set up a custom PTR record or delegate the record to 
someone else.  Most of these static customers are legitimate business 
networks running their own mail server, and have neither the need nor 
desire to relay their mail through Comcast's SMTP servers.  I think your 
general idea is very good, but you're reaching a little too far with 
this one.


I think based on all of the feedback I'm getting on this, I'm going to 
have a config option for something like 
relaychecker_skip_statichostname=1 with 1 being the default.  It will 
cause both the IP in hostname and dynamic hostname checks to be 
skipped if \bstatic\b is in the hostname.  I may also have a 
relaychecker_skip_iphostname and relaychecker_skip_dynamichostname, 
which default to 0 ... to allow places like Italian sites to skip those 
entirely if they just want the basic DNS checks.


It may be a couple days before I can make the changes I've put 
forward... we're having a problem at work (not related to this; it's at 
the network level), and I wont be able to put much coding/testing time 
into this until that problem gets handled.



John

Re: Relay Checker Plugin (code review please?)

[Matthew Newton]

Hi,

On Mon, Oct 30, 2006 at 03:23:21PM -0800, John Rudd wrote:
 I've written a plugin for Spam Assassin that does the relay checks I 

...and here was me just working out how to get exim to check this,
and have SpamAssassin add a score, and your mail arrived :-)

 1) no RDNS for the machines that aren't intended to talk to the outside 
 world
 
 2) RDNS that doesn't lead back to a valid A record
 
 3) RDNS that is forged (leads to an A record which doesn't resolve back 
 to the IP you started with)
 
 4) Contains the hosts IP address within the hostname
 
 5) Contains standard key words within the hostname (but not in the TLD 
 nor registered domain name), such as dhcp, dialup, dial-up, dsl, 
 etc.

I'm also thinking about connections that use one of these I'm on
an ADSL line-type names for the HELO string. Not directly
rejecting, again, just adding a score on.

If this really was just home connections, then I'd block directly.
As there are some legitimate businesses (with braindead ISPs) as
already pointed out, adding an extra score shouldn't matter for
them (unless they actually are sending spam, which is a different
matter altogether).

 The two files you need (put them in /etc/mail/spamassassin ... or 
 wherever you want to put your plugins) are:

I'll drop it on our mailers (probably with a smaller score than
the default) and let you know how many times the phone rings
before I have to tweak it or remove it ;-).

Matthew


-- 
Matthew Newton [EMAIL PROTECTED]

UNIX and e-mail Systems Administrator, Network Support Section,
Computer Centre, University of Leicester,
Leicester LE1 7RH, United Kingdom

Re: R: R: R: Relay Checker Plugin (code review please?)

[Steven Dickenson]

On Oct 31, 2006, at 6:09 AM, John Rudd wrote:

I've considered the exact opposite (adding static to the check for  
keywords).  My rules are really looking more for is this a  
_client_ host, not is this a dynamic host.  That one check looks  
for dynamic, but I'm not interested in exempting anyone because  
they're static.  They've still got a hostname that looks like an  
end-client, and an end-client shouldn't be connecting to other  
people's mail servers.  Any end-client that connects to someone  
else's email server should be treated like it's a spam/virus zombie


I can't agree with this.  Many small businesses in the US get just  
these kind of static connections from broadband ISPs.  Comcast, for  
example, has all of their static customers using rDNS that would fail  
your tests, and they refuse to set up a custom PTR record or delegate  
the record to someone else.  Most of these static customers are  
legitimate business networks running their own mail server, and have  
neither the need nor desire to relay their mail through Comcast's  
SMTP servers.  I think your general idea is very good, but you're  
reaching a little too far with this one.


Steven
---
Steven Dickenson [EMAIL PROTECTED]
http://www.mrchuckles.net

Re: Relay Checker Plugin (code review please?)

[John Rudd]

Stuart Johnston wrote:

John Rudd wrote:



2) This sort of replaces the other set of rules I created, that did 
this with metarules instead of a plugin.  This made some of the checks 
less useful.  You probably don't need to use both methods.


So, what is the point of doing this as a plugin instead of using 
existing rules?  The obvious disadvantage is the additional dns lookups.


The advantages are:

a) being sure that the hostname in RDNS points back to the IP address 
you started with.  Thus detecting forgeries (which shouldn't happen with 
_any_ legitimate service)


b) just using the rules version of what I wrote, you can only check if 
the decimal IP address, in individual segments, is in the hostname.  You 
can't check if the entire decimal IP address (one large number) is in 
the IP address, nor can you check if the hexidecimal segments are in the 
hostname.



(a) requires more DNS work, yes.  (b) does not.  It just requires a bit 
more math.

Re: Relay Checker Plugin (code review please?)

[Stuart Johnston]

John Rudd wrote:

Stuart Johnston wrote:

John Rudd wrote:



2) This sort of replaces the other set of rules I created, that did 
this with metarules instead of a plugin.  This made some of the 
checks less useful.  You probably don't need to use both methods.


So, what is the point of doing this as a plugin instead of using 
existing rules?  The obvious disadvantage is the additional dns lookups.


The advantages are:

a) being sure that the hostname in RDNS points back to the IP address 
you started with.  Thus detecting forgeries (which shouldn't happen with 
_any_ legitimate service)


Postfix does this for you.  It is easy enough to write an SA rule to look at the Postfix headers.  I 
don't know about other MTAs.



b) just using the rules version of what I wrote, you can only check if 
the decimal IP address, in individual segments, is in the hostname.  You 
can't check if the entire decimal IP address (one large number) is in 
the IP address, nor can you check if the hexidecimal segments are in the 
hostname.



(a) requires more DNS work, yes.  (b) does not.  It just requires a bit 
more math.




This is just my opinion, of course, but:  I'd probably make the plugin just do 
(b).

It might be nice if SA did (a) as part of its standard checks although in my experience, way too 
many legitimate mail servers fail on this for it to be useful anyway.

Re: Relay Checker Plugin (code review please?)

[John Rudd]

Stuart Johnston wrote:

John Rudd wrote:

Stuart Johnston wrote:

John Rudd wrote:



2) This sort of replaces the other set of rules I created, that did 
this with metarules instead of a plugin.  This made some of the 
checks less useful.  You probably don't need to use both methods.


So, what is the point of doing this as a plugin instead of using 
existing rules?  The obvious disadvantage is the additional dns lookups.


The advantages are:

a) being sure that the hostname in RDNS points back to the IP address 
you started with.  Thus detecting forgeries (which shouldn't happen 
with _any_ legitimate service)


Postfix does this for you.  It is easy enough to write an SA rule to 
look at the Postfix headers.  I don't know about other MTAs.


Sendmail does some of it, but since I didn't find detailed documentation 
on the Trusted/Untrusted Relay pseudo-headers, I don't know if its 
represented in there.  Nor do I know if it's on the meta-information I 
can get from permessagestatus when I ask for the untrusted relay entries 
(whose hash keys are, I assume, the names of the fields in the 
trusted/untrusted relays lines)


If I could get that same information without the DNS checks, I would. 
(though, honestly, with a little more investigation, I can probably 
eliminate ONE of my two DNS checks by looking at more of the pseudo-header).



b) just using the rules version of what I wrote, you can only check if 
the decimal IP address, in individual segments, is in the hostname.  
You can't check if the entire decimal IP address (one large number) is 
in the IP address, nor can you check if the hexidecimal segments are 
in the hostname.



(a) requires more DNS work, yes.  (b) does not.  It just requires a 
bit more math.




This is just my opinion, of course, but:  I'd probably make the plugin 
just do (b).


It might be nice if SA did (a) as part of its standard checks although 
in my experience, way too many legitimate mail servers fail on this for 
it to be useful anyway.


I have yet to have a legitimate message rejected by that check, when 
I've been doing it in mimedefang.

Relay Checker Plugin (code review please?)

[John Rudd]

I've written a plugin for Spam Assassin that does the relay checks I 
used to do in MimeDefang.  The purpose of these checks is to try to 
identify those messages that are likely to be coming directly (with no 
intermediary mail server) from a zombie-bot, and are thus likely to be 
spam (or maybe virus) content.  It does this by looking at 
characteristics of how ISPs and large networks tend to layout the 
hostnames of their dynamic hosts and end clients.  This includes:


1) no RDNS for the machines that aren't intended to talk to the outside 
world


2) RDNS that doesn't lead back to a valid A record

3) RDNS that is forged (leads to an A record which doesn't resolve back 
to the IP you started with)


4) Contains the hosts IP address within the hostname

5) Contains standard key words within the hostname (but not in the TLD 
nor registered domain name), such as dhcp, dialup, dial-up, dsl, 
etc.



From this, a score of 5 or 6 is generated (it's really 4+ number of 
checks failed, but several of the checks are mutually exclusive).  This 
should be enough to flag the message for review/quarantine, but not 
enough to automatically delete or reject the message (because none of 
you are doing that at a score of 5 or 6, right? right.).  Thus, a false 
positive will merely result in a quarantine situation.


In my own results, I have seen this to be HIGHLY accurate.  I have yet 
to get a false positive ... but it has caught several types of spam that 
other methods simply haven't been able to catch (or require significant 
processing, such as OCR, to catch).


The two files you need (put them in /etc/mail/spamassassin ... or 
wherever you want to put your plugins) are:


http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.cf
http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.pm


some notes:

1) I don't use Net::DNS for my checks, I use the built in perl get* 
calls. Mostly because I haven't looked at Net::DNS yet.  If someone 
wanted to submit that code change to me, I'd gladly look at it.  I'll 
get to it eventually on my own, though (might as well; SA already uses 
Net::DNS, right?).


2) This sort of replaces the other set of rules I created, that did this 
with metarules instead of a plugin.  This made some of the checks less 
useful.  You probably don't need to use both methods.


3) for those who object to SA checks that aren't purely about message 
content, you wont like this plugin.  It's about trying to remove a class 
of sender (spambots, and mis-configured clients that aren't using their 
own domain's mail server for outbound traffic), where that class of 
sender is OVERWHELMINGLY likely to be generating spam.  Just like open 
relays are overwhelmingly likely to be generating spam.  My hope is that 
it may eliminate, or severely reduce, the spambot problem: this is a 
feature of the sending machine that the spammer and bot-master have _NO_ 
control over, so they can't adjust their content nor behaviors to adapt 
to it.  They would simply have to give up using systems whose DNS 
configuration matche these tests.



So, if people could take a look at it, test it, see if it does what it 
advertises, and see if it's as accurate as my experience indicates, I 
would appreciate getting feedback.  If it pans out, I'll see about 
putting it in a tar ball, and submitting it to the wiki's list of plugins.



John

RE: Relay Checker Plugin (code review please?)

[Dylan Bouterse]

-Original Message-
From: John Rudd [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 6:23 PM
To: SpamAssassin Users
Subject: Relay Checker Plugin (code review please?)


I've written a plugin for Spam Assassin that does the relay checks I 
used to do in MimeDefang.  The purpose of these checks is to try to 
identify those messages that are likely to be coming directly (with no 
intermediary mail server) from a zombie-bot, and are thus likely to be 
spam (or maybe virus) content.  It does this by looking at 
characteristics of how ISPs and large networks tend to layout the 
hostnames of their dynamic hosts and end clients.  This includes:

1) no RDNS for the machines that aren't intended to talk to the outside 
world

2) RDNS that doesn't lead back to a valid A record

3) RDNS that is forged (leads to an A record which doesn't resolve back 
to the IP you started with)

4) Contains the hosts IP address within the hostname

5) Contains standard key words within the hostname (but not in the TLD 
nor registered domain name), such as dhcp, dialup, dial-up, dsl,

etc.


 From this, a score of 5 or 6 is generated (it's really 4+ number of 
checks failed, but several of the checks are mutually exclusive).  This 
should be enough to flag the message for review/quarantine, but not 
enough to automatically delete or reject the message (because none of 
you are doing that at a score of 5 or 6, right? right.).  Thus, a false 
positive will merely result in a quarantine situation.

In my own results, I have seen this to be HIGHLY accurate.  I have yet 
to get a false positive ... but it has caught several types of spam that

other methods simply haven't been able to catch (or require significant 
processing, such as OCR, to catch).

The two files you need (put them in /etc/mail/spamassassin ... or 
wherever you want to put your plugins) are:

http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.cf
http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.pm


some notes:

1) I don't use Net::DNS for my checks, I use the built in perl get* 
calls. Mostly because I haven't looked at Net::DNS yet.  If someone 
wanted to submit that code change to me, I'd gladly look at it.  I'll 
get to it eventually on my own, though (might as well; SA already uses 
Net::DNS, right?).

2) This sort of replaces the other set of rules I created, that did this

with metarules instead of a plugin.  This made some of the checks less 
useful.  You probably don't need to use both methods.

3) for those who object to SA checks that aren't purely about message 
content, you wont like this plugin.  It's about trying to remove a class

of sender (spambots, and mis-configured clients that aren't using their 
own domain's mail server for outbound traffic), where that class of 
sender is OVERWHELMINGLY likely to be generating spam.  Just like open 
relays are overwhelmingly likely to be generating spam.  My hope is that

it may eliminate, or severely reduce, the spambot problem: this is a 
feature of the sending machine that the spammer and bot-master have _NO_

control over, so they can't adjust their content nor behaviors to adapt 
to it.  They would simply have to give up using systems whose DNS 
configuration matche these tests.


So, if people could take a look at it, test it, see if it does what it 
advertises, and see if it's as accurate as my experience indicates, I 
would appreciate getting feedback.  If it pans out, I'll see about 
putting it in a tar ball, and submitting it to the wiki's list of
plugins.


John



How would one adjust the score down for testing purposes?

Dylan

Re: Relay Checker Plugin (code review please?)

[John Rudd]

Dylan Bouterse wrote:


-Original Message-
From: John Rudd [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 6:23 PM

To: SpamAssassin Users
Subject: Relay Checker Plugin (code review please?)


I've written a plugin for Spam Assassin that does the relay checks I 
used to do in MimeDefang.  The purpose of these checks is to try to 
identify those messages that are likely to be coming directly (with no 
intermediary mail server) from a zombie-bot, and are thus likely to be 
spam (or maybe virus) content.  It does this by looking at 
characteristics of how ISPs and large networks tend to layout the 
hostnames of their dynamic hosts and end clients.  This includes:


1) no RDNS for the machines that aren't intended to talk to the outside 
world


2) RDNS that doesn't lead back to a valid A record

3) RDNS that is forged (leads to an A record which doesn't resolve back 
to the IP you started with)


4) Contains the hosts IP address within the hostname

5) Contains standard key words within the hostname (but not in the TLD 
nor registered domain name), such as dhcp, dialup, dial-up, dsl,


etc.


 From this, a score of 5 or 6 is generated (it's really 4+ number of 
checks failed, but several of the checks are mutually exclusive).  This 
should be enough to flag the message for review/quarantine, but not 
enough to automatically delete or reject the message (because none of 
you are doing that at a score of 5 or 6, right? right.).  Thus, a false 
positive will merely result in a quarantine situation.


In my own results, I have seen this to be HIGHLY accurate.  I have yet 
to get a false positive ... but it has caught several types of spam that


other methods simply haven't been able to catch (or require significant 
processing, such as OCR, to catch).


The two files you need (put them in /etc/mail/spamassassin ... or 
wherever you want to put your plugins) are:


http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.cf
http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.pm


some notes:

1) I don't use Net::DNS for my checks, I use the built in perl get* 
calls. Mostly because I haven't looked at Net::DNS yet.  If someone 
wanted to submit that code change to me, I'd gladly look at it.  I'll 
get to it eventually on my own, though (might as well; SA already uses 
Net::DNS, right?).


2) This sort of replaces the other set of rules I created, that did this

with metarules instead of a plugin.  This made some of the checks less 
useful.  You probably don't need to use both methods.


3) for those who object to SA checks that aren't purely about message 
content, you wont like this plugin.  It's about trying to remove a class


of sender (spambots, and mis-configured clients that aren't using their 
own domain's mail server for outbound traffic), where that class of 
sender is OVERWHELMINGLY likely to be generating spam.  Just like open 
relays are overwhelmingly likely to be generating spam.  My hope is that


it may eliminate, or severely reduce, the spambot problem: this is a 
feature of the sending machine that the spammer and bot-master have _NO_


control over, so they can't adjust their content nor behaviors to adapt 
to it.  They would simply have to give up using systems whose DNS 
configuration matche these tests.



So, if people could take a look at it, test it, see if it does what it 
advertises, and see if it's as accurate as my experience indicates, I 
would appreciate getting feedback.  If it pans out, I'll see about 
putting it in a tar ball, and submitting it to the wiki's list of

plugins.


John



How would one adjust the score down for testing purposes?

Dylan


You'd have to adjust the source.  The score determination is right 
around/before the last big if clause.

Re: Relay Checker Plugin (code review please?)

[Rick Macdougall]

John Rudd wrote:


I've written a plugin for Spam Assassin that does the relay checks I 
used to do in MimeDefang.  The purpose of these checks is to try to 
identify those messages that are likely to be coming directly (with no 
intermediary mail server) from a zombie-bot, and are thus likely to be 
spam (or maybe virus) content.  It does this by looking at 
characteristics of how ISPs and large networks tend to layout the 
hostnames of their dynamic hosts and end clients.  This includes:


1) no RDNS for the machines that aren't intended to talk to the outside 
world


2) RDNS that doesn't lead back to a valid A record

3) RDNS that is forged (leads to an A record which doesn't resolve back 
to the IP you started with)


4) Contains the hosts IP address within the hostname

5) Contains standard key words within the hostname (but not in the TLD 
nor registered domain name), such as dhcp, dialup, dial-up, dsl, 
etc.



 From this, a score of 5 or 6 is generated (it's really 4+ number of 
checks failed, but several of the checks are mutually exclusive).  This 
should be enough to flag the message for review/quarantine, but not 
enough to automatically delete or reject the message (because none of 
you are doing that at a score of 5 or 6, right? right.).  Thus, a false 
positive will merely result in a quarantine situation.


In my own results, I have seen this to be HIGHLY accurate.  I have yet 
to get a false positive ... but it has caught several types of spam that 
other methods simply haven't been able to catch (or require significant 
processing, such as OCR, to catch).


The two files you need (put them in /etc/mail/spamassassin ... or 
wherever you want to put your plugins) are:


http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.cf
http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.pm


some notes:

1) I don't use Net::DNS for my checks, I use the built in perl get* 
calls. Mostly because I haven't looked at Net::DNS yet.  If someone 
wanted to submit that code change to me, I'd gladly look at it.  I'll 
get to it eventually on my own, though (might as well; SA already uses 
Net::DNS, right?).


2) This sort of replaces the other set of rules I created, that did this 
with metarules instead of a plugin.  This made some of the checks less 
useful.  You probably don't need to use both methods.


3) for those who object to SA checks that aren't purely about message 
content, you wont like this plugin.  It's about trying to remove a class 
of sender (spambots, and mis-configured clients that aren't using their 
own domain's mail server for outbound traffic), where that class of 
sender is OVERWHELMINGLY likely to be generating spam.  Just like open 
relays are overwhelmingly likely to be generating spam.  My hope is that 
it may eliminate, or severely reduce, the spambot problem: this is a 
feature of the sending machine that the spammer and bot-master have _NO_ 
control over, so they can't adjust their content nor behaviors to adapt 
to it.  They would simply have to give up using systems whose DNS 
configuration matche these tests.



So, if people could take a look at it, test it, see if it does what it 
advertises, and see if it's as accurate as my experience indicates, I 
would appreciate getting feedback.  If it pans out, I'll see about 
putting it in a tar ball, and submitting it to the wiki's list of plugins.





Hi,

Running it now on one of our 4 SA servers.  I'll get back to you in the 
morning about how well it does.


Regards,

Rick

Re: Relay Checker Plugin (code review please?)

[Rick Macdougall]

John Rudd wrote:


I've written a plugin for Spam Assassin that does the relay checks I 
used to do in MimeDefang.  The purpose of these checks is to try to 
identify those messages that are likely to be coming directly (with no 
intermediary mail server) from a zombie-bot, and are thus likely to be 
spam (or maybe virus) content.  It does this by looking at 
characteristics of how ISPs and large networks tend to layout the 
hostnames of their dynamic hosts and end clients.  This includes:


1) no RDNS for the machines that aren't intended to talk to the outside 
world


2) RDNS that doesn't lead back to a valid A record

3) RDNS that is forged (leads to an A record which doesn't resolve back 
to the IP you started with)


4) Contains the hosts IP address within the hostname

5) Contains standard key words within the hostname (but not in the TLD 
nor registered domain name), such as dhcp, dialup, dial-up, dsl, 
etc.



 From this, a score of 5 or 6 is generated (it's really 4+ number of 
checks failed, but several of the checks are mutually exclusive).  This 
should be enough to flag the message for review/quarantine, but not 
enough to automatically delete or reject the message (because none of 
you are doing that at a score of 5 or 6, right? right.).  Thus, a false 
positive will merely result in a quarantine situation.


In my own results, I have seen this to be HIGHLY accurate.  I have yet 
to get a false positive ... but it has caught several types of spam that 
other methods simply haven't been able to catch (or require significant 
processing, such as OCR, to catch).


The two files you need (put them in /etc/mail/spamassassin ... or 
wherever you want to put your plugins) are:


http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.cf
http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.pm


some notes:

1) I don't use Net::DNS for my checks, I use the built in perl get* 
calls. Mostly because I haven't looked at Net::DNS yet.  If someone 
wanted to submit that code change to me, I'd gladly look at it.  I'll 
get to it eventually on my own, though (might as well; SA already uses 
Net::DNS, right?).


2) This sort of replaces the other set of rules I created, that did this 
with metarules instead of a plugin.  This made some of the checks less 
useful.  You probably don't need to use both methods.


3) for those who object to SA checks that aren't purely about message 
content, you wont like this plugin.  It's about trying to remove a class 
of sender (spambots, and mis-configured clients that aren't using their 
own domain's mail server for outbound traffic), where that class of 
sender is OVERWHELMINGLY likely to be generating spam.  Just like open 
relays are overwhelmingly likely to be generating spam.  My hope is that 
it may eliminate, or severely reduce, the spambot problem: this is a 
feature of the sending machine that the spammer and bot-master have _NO_ 
control over, so they can't adjust their content nor behaviors to adapt 
to it.  They would simply have to give up using systems whose DNS 
configuration matche these tests.



So, if people could take a look at it, test it, see if it does what it 
advertises, and see if it's as accurate as my experience indicates, I 
would appreciate getting feedback.  If it pans out, I'll see about 
putting it in a tar ball, and submitting it to the wiki's list of plugins.


Hi,

Right off the bat I've disabled it.  It, of course, hits on all mail my 
local users send.  That's not really acceptable in an ISP situation so 
I've turned it off until tomorrow when I have the time to look at the 
code and see if I can disable the check for specific IP's or host names.


I can say it was hitting on a lot of spam that was passing through as 
clean before, so there is quite a bit of merit to the idea.  It would 
just need the ability to ignore local clients.


Regards,

Rick

Re: Relay Checker Plugin (code review please?)

[John Rudd]

Rick Macdougall wrote:

John Rudd wrote:


I've written a plugin for Spam Assassin that does the relay checks I 
used to do in MimeDefang.  The purpose of these checks is to try to 
identify those messages that are likely to be coming directly (with no 
intermediary mail server) from a zombie-bot, and are thus likely to be 
spam (or maybe virus) content.  It does this by looking at 
characteristics of how ISPs and large networks tend to layout the 
hostnames of their dynamic hosts and end clients.  This includes:


1) no RDNS for the machines that aren't intended to talk to the 
outside world


2) RDNS that doesn't lead back to a valid A record

3) RDNS that is forged (leads to an A record which doesn't resolve 
back to the IP you started with)


4) Contains the hosts IP address within the hostname

5) Contains standard key words within the hostname (but not in the TLD 
nor registered domain name), such as dhcp, dialup, dial-up, 
dsl, etc.



 From this, a score of 5 or 6 is generated (it's really 4+ number of 
checks failed, but several of the checks are mutually exclusive).  
This should be enough to flag the message for review/quarantine, but 
not enough to automatically delete or reject the message (because none 
of you are doing that at a score of 5 or 6, right? right.).  Thus, a 
false positive will merely result in a quarantine situation.


In my own results, I have seen this to be HIGHLY accurate.  I have yet 
to get a false positive ... but it has caught several types of spam 
that other methods simply haven't been able to catch (or require 
significant processing, such as OCR, to catch).


The two files you need (put them in /etc/mail/spamassassin ... or 
wherever you want to put your plugins) are:


http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.cf
http://people.ucsc.edu/~jrudd/spamassassin/RelayChecker.pm


some notes:

1) I don't use Net::DNS for my checks, I use the built in perl get* 
calls. Mostly because I haven't looked at Net::DNS yet.  If someone 
wanted to submit that code change to me, I'd gladly look at it.  I'll 
get to it eventually on my own, though (might as well; SA already uses 
Net::DNS, right?).


2) This sort of replaces the other set of rules I created, that did 
this with metarules instead of a plugin.  This made some of the checks 
less useful.  You probably don't need to use both methods.


3) for those who object to SA checks that aren't purely about message 
content, you wont like this plugin.  It's about trying to remove a 
class of sender (spambots, and mis-configured clients that aren't 
using their own domain's mail server for outbound traffic), where that 
class of sender is OVERWHELMINGLY likely to be generating spam.  Just 
like open relays are overwhelmingly likely to be generating spam.  My 
hope is that it may eliminate, or severely reduce, the spambot 
problem: this is a feature of the sending machine that the spammer and 
bot-master have _NO_ control over, so they can't adjust their content 
nor behaviors to adapt to it.  They would simply have to give up using 
systems whose DNS configuration matche these tests.



So, if people could take a look at it, test it, see if it does what it 
advertises, and see if it's as accurate as my experience indicates, I 
would appreciate getting feedback.  If it pans out, I'll see about 
putting it in a tar ball, and submitting it to the wiki's list of 
plugins.


Hi,

Right off the bat I've disabled it.  It, of course, hits on all mail my 
local users send.  That's not really acceptable in an ISP situation so 
I've turned it off until tomorrow when I have the time to look at the 
code and see if I can disable the check for specific IP's or host names.


I can say it was hitting on a lot of spam that was passing through as 
clean before, so there is quite a bit of merit to the idea.  It would 
just need the ability to ignore local clients.




Are those users on your trusted network?  It should only be looking at 
your first untrusted relay.


Though, if they're authenticated, I wouldn't mind trying to figure out 
how to extract that from the information, and exempt those.


I could easily add a list of exemptions though.

R: Relay Checker Plugin (code review please?)

[Giampaolo Tomassoni]

 So, if people could take a look at it, test it, see if it does what it 
 advertises, and see if it's as accurate as my experience indicates, I 
 would appreciate getting feedback.  If it pans out, I'll see about 
 putting it in a tar ball, and submitting it to the wiki's list of plugins.

I didn't yet manage to test it, but it looks like an interesting work.

May I spare a suggestion?

I would prefer not to have to deal with a single, computed RELAY_CHECKER score, 
but with many different ones for each of the triggered cases. This way it would 
be easier to tune scores from this plugin.

To me, your plugin could trigger the following tags:

RELAY_CHECKER (at least one rule had been triggered. According to your code 
would score 4 by default);
RC_NORDNS (scores 1);
RC_BADRDNS  (scores 1);
RC_BADDNS   (scores 1);
RC_IPINHOSTNAME (scores 1);
RC_DYNHOSTNAME  (scores 1);

---
Giampaolo Tomassoni - IT Consultant
Piazza VIII Aprile 1948, 4
I-53044 Chiusi (SI) - Italy
Ph: +39-0578-21100

MAI inviare una e-mail a:
NEVER send an e-mail to:
 [EMAIL PROTECTED]

 
 
 John