Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-16 Thread Mike K
May want to account for foreign languages also. I just received this spam
while I was adding your URL obfuscation filter.

#1053;#1077;#1076;#1086;#1088;#1086;#1075;#1080;#1077;
#1079;#1074;#1086;#1085;#1082;#1080;
#1079;#1072;#1088;#1091;#1073;#1077;#1078;!

Mike


- Original Message -
From: Matthew Bramble [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, September 15, 2003 12:40 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 Pete,

 It's not redundant because the two by themselves only check for strings
 of two, while the combination checks for strings with one of each in
 succession.  This way, if they go back and forth between the two, it
 will get caught as long as there is a . or @ between them, or as
 long as it is URL encoding followed by HTML encoding.  I left out the
 other way around because it was only a two character string, ;% and
 wanted to protect from FP's.

 I do appreciate the feedback though...I do of course make mistakes.

 Matt

 Pete McNeil wrote:

  Matt,
 
  It appears that your coding for a combination of http  url encoding
  in urls is redundant since you capture both types individually. It's a
  small optimization, but worth mentioning.
 
  _M
 
  At 07:46 PM 9/14/2003 -0400, you wrote:
 
  I've posted a newer version of the OBFUSCATION filter on my site.
  This contains the removal of the attachment thing and also the
  removal of 6 (of over 100) tests in order to be more forgiving, sans
  the PayPal issue.
 
 
http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003c.txt
 
 
  If you find any false positives with this besides the Ticketmaster
  one that I've already counterbalanced, please let me know.  I would
  imagine that posting to this group would be better than PM's unless
  others mind having discussion here.  That way everyone would know
  about any issues ASAP.
 
  Thanks,
 
  Matt
 
  ---
  [This E-mail was scanned for viruses by Declude Virus
  (http://www.declude.com)]
 
  ---
  This E-mail came from the Declude.JunkMail mailing list.  To
  unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
  type unsubscribe Declude.JunkMail.  The archives can be found
  at http://www.mail-archive.com.
 
 
  ---
  [This E-mail was scanned for viruses by Declude Virus
  (http://www.declude.com)]
 
  ---
  This E-mail came from the Declude.JunkMail mailing list.  To
  unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
  type unsubscribe Declude.JunkMail.  The archives can be found
  at http://www.mail-archive.com.


 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.




---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-16 Thread Mike K
Sorry, just noticed, this was in the subject.

Mike

- Original Message -
From: Mike K [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Tuesday, September 16, 2003 3:32 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 May want to account for foreign languages also. I just received this spam
 while I was adding your URL obfuscation filter.

 #1053;#1077;#1076;#1086;#1088;#1086;#1075;#1080;#1077;
 #1079;#1074;#1086;#1085;#1082;#1080;
 #1079;#1072;#1088;#1091;#1073;#1077;#1078;!

 Mike


 - Original Message -
 From: Matthew Bramble [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Sent: Monday, September 15, 2003 12:40 PM
 Subject: Re: [Declude.JunkMail] OBFUSCATION filter


  Pete,
 
  It's not redundant because the two by themselves only check for strings
  of two, while the combination checks for strings with one of each in
  succession.  This way, if they go back and forth between the two, it
  will get caught as long as there is a . or @ between them, or as
  long as it is URL encoding followed by HTML encoding.  I left out the
  other way around because it was only a two character string, ;% and
  wanted to protect from FP's.
 
  I do appreciate the feedback though...I do of course make mistakes.
 
  Matt
 
  Pete McNeil wrote:
 
   Matt,
  
   It appears that your coding for a combination of http  url encoding
   in urls is redundant since you capture both types individually. It's a
   small optimization, but worth mentioning.
  
   _M
  
   At 07:46 PM 9/14/2003 -0400, you wrote:
  
   I've posted a newer version of the OBFUSCATION filter on my site.
   This contains the removal of the attachment thing and also the
   removal of 6 (of over 100) tests in order to be more forgiving, sans
   the PayPal issue.
  
  
 http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003c.txt
  
  
   If you find any false positives with this besides the Ticketmaster
   one that I've already counterbalanced, please let me know.  I would
   imagine that posting to this group would be better than PM's unless
   others mind having discussion here.  That way everyone would know
   about any issues ASAP.
  
   Thanks,
  
   Matt
  
   ---
   [This E-mail was scanned for viruses by Declude Virus
   (http://www.declude.com)]
  
   ---
   This E-mail came from the Declude.JunkMail mailing list.  To
   unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
   type unsubscribe Declude.JunkMail.  The archives can be found
   at http://www.mail-archive.com.
  
  
   ---
   [This E-mail was scanned for viruses by Declude Virus
   (http://www.declude.com)]
  
   ---
   This E-mail came from the Declude.JunkMail mailing list.  To
   unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
   type unsubscribe Declude.JunkMail.  The archives can be found
   at http://www.mail-archive.com.
 
 
  ---
  [This E-mail was scanned for viruses by Declude Virus
 (http://www.declude.com)]
 
  ---
  This E-mail came from the Declude.JunkMail mailing list.  To
  unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
  type unsubscribe Declude.JunkMail.  The archives can be found
  at http://www.mail-archive.com.
 
 


 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.




---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-16 Thread Matthew Bramble
Mike,

The same thing can happen in the body, so it's worth knowing.  Naturally 
the filter can easily be modified for use in the subject, and there is 
really no reason at all to be HTML encoding subject lines unless it is a 
non-Western European language, and still they should be base64 encoded I 
would think.  I don't think the URL encoding techniques need be applied 
to subjects though, but searching a subject shouldn't be that process 
intensive.

Matt

Mike K wrote:

Sorry, just noticed, this was in the subject.

Mike

- Original Message -
From: Mike K [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Tuesday, September 16, 2003 3:32 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter
 

May want to account for foreign languages also. I just received this spam
while I was adding your URL obfuscation filter.
#1053;#1077;#1076;#1086;#1088;#1086;#1075;#1080;#1077;
#1079;#1074;#1086;#1085;#1082;#1080;
#1079;#1072;#1088;#1091;#1073;#1077;#1078;!
Mike

- Original Message -
From: Matthew Bramble [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, September 15, 2003 12:40 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter
   

Pete,

It's not redundant because the two by themselves only check for strings
of two, while the combination checks for strings with one of each in
succession.  This way, if they go back and forth between the two, it
will get caught as long as there is a . or @ between them, or as
long as it is URL encoding followed by HTML encoding.  I left out the
other way around because it was only a two character string, ;% and
wanted to protect from FP's.
I do appreciate the feedback though...I do of course make mistakes.

Matt

Pete McNeil wrote:

 

Matt,

It appears that your coding for a combination of http  url encoding
in urls is redundant since you capture both types individually. It's a
small optimization, but worth mentioning.
_M

At 07:46 PM 9/14/2003 -0400, you wrote:

   

I've posted a newer version of the OBFUSCATION filter on my site.
This contains the removal of the attachment thing and also the
removal of 6 (of over 100) tests in order to be more forgiving, sans
the PayPal issue.
 

http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003c.txt
   

If you find any false positives with this besides the Ticketmaster
one that I've already counterbalanced, please let me know.  I would
imagine that posting to this group would be better than PM's unless
others mind having discussion here.  That way everyone would know
about any issues ASAP.
Thanks,

Matt
 



---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-16 Thread Matthew Bramble




Mike,

Good point, however there is a problem. What you have is HTML encoded
UNICODE, and there are thousands upon thousands of these:
http://www.alanwood.net/unicode/unicode_samples_no.html , and there
might be a good reason for this in multi-lingual mailings. I don't
think though that mail clients would be supporting this method because
base64 encoding is a lot more efficient with the overhead than HTML
encoding is.

You could potentially test for just ";#" in order to find two HTML
encoded characters of any type in succession, however there are valid
uses where you are listing two symbols in succession and the FP's would
probably come into play. Such examples would probably be rare, so if
you score the filter low in the first place, this wouldn't have a big
impact. Adding that three character string would also defeat the need
for 62 of the BODY checks in that filter and save on some processing, I
just don't know that it would be safe to do.

If someone with a decent mail volume and a decent number of clients
that have foreign language customers would like to test this for FP's
and let the list know, that would be valuable. The filter would be the
following:
-Global.cfg-
HTMLENCODE-TEST filter 
C:\IMail\Declude\Filters\HTMLEncode-Test.txt x 0  0
  
-HTMLEncode-Test.txt-
BODY  0 CONTAINS ;#
  
-$Default$.JunkMail-
HTMLENCODE-TEST COPYTO [EMAIL PROTECTED]


I don't think my volume is large enough to get a feeling for the
potential of FP's from this modification. The existing filter though
should hardly ever get an FP.

Matt



Mike K wrote:

  May want to account for foreign languages also. I just received this spam
while I was adding your URL obfuscation filter.

#1053;#1077;#1076;#1086;#1088;#1086;#1075;#1080;#1077;
#1079;#1074;#1086;#1085;#1082;#1080;
#1079;#1072;#1088;#1091;#1073;#1077;#1078;!

Mike


- Original Message -
From: "Matthew Bramble" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, September 15, 2003 12:40 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


  
  
Pete,

It's not redundant because the two by themselves only check for strings
of two, while the combination checks for strings with one of each in
succession.  This way, if they go back and forth between the two, it
will get caught as long as there is a "." or "@" between them, or as
long as it is URL encoding followed by HTML encoding.  I left out the
other way around because it was only a two character string, ";%" and
wanted to protect from FP's.

I do appreciate the feedback though...I do of course make mistakes.

Matt

Pete McNeil wrote:



  Matt,

It appears that your coding for a combination of http  url encoding
in urls is redundant since you capture both types individually. It's a
small optimization, but worth mentioning.

_M

At 07:46 PM 9/14/2003 -0400, you wrote:

  
  
I've posted a newer version of the OBFUSCATION filter on my site.
This contains the removal of the attachment thing and also the
removal of 6 (of over 100) tests in order to be more forgiving, sans
the PayPal issue.



  

  
  http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003c.txt
  
  

  

If you find any false positives with this besides the Ticketmaster
one that I've already counterbalanced, please let me know.  I would
imagine that posting to this group would be better than PM's unless
others mind having discussion here.  That way everyone would know
about any issues ASAP.

Thanks,

Matt

  

  






Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Pete McNeil
Matt,

It appears that your coding for a combination of http  url encoding in 
urls is redundant since you capture both types individually. It's a small 
optimization, but worth mentioning.

_M

At 07:46 PM 9/14/2003 -0400, you wrote:
I've posted a newer version of the OBFUSCATION filter on my site.  This 
contains the removal of the attachment thing and also the removal of 6 (of 
over 100) tests in order to be more forgiving, sans the PayPal issue.

http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003c.txt

If you find any false positives with this besides the Ticketmaster one 
that I've already counterbalanced, please let me know.  I would imagine 
that posting to this group would be better than PM's unless others mind 
having discussion here.  That way everyone would know about any issues ASAP.

Thanks,

Matt

---
[This E-mail was scanned for viruses by Declude Virus 
(http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.
---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Pete McNeil
At 05:58 AM 9/15/2003 -0400, you wrote:
Matt,

It appears that your coding for a combination of http  url encoding in 
urls is redundant since you capture both types individually. It's a small 
optimization, but worth mentioning.

_M
ooops.. Sorry, I meant html. 

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


RE: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Kami Razvan
Hi Bill:
You are right...  No disagreement here.

We had negative MAILFROM but it was being abused like crazy.  We were
getting so much spam from faked addresses.  We now have a negative list for
mailing lists and at times we see email coming through.

REVDNS whitelist has worked well and we have not yet seen any abuses - but
as a rule I agree with you it can be abused.

Since someone asked about our whitelist- here it is (these are the general
items - we have in this list some of our clients with screwed up server
setups but are taken out in this list).  This goes in the Global.cfg file.

WHITELIST   REVDNS  .airborne.com
WHITELIST   REVDNS  .amazon.com
WHITELIST   REVDNS  .audible.com
WHITELIST   REVDNS  .bestfares.com
WHITELIST   REVDNS  .cnet.com
WHITELIST   REVDNS  .dell.com
WHITELIST   REVDNS  .dowjones.com
WHITELIST   REVDNS  .ebay.com
WHITELIST   REVDNS  .equifax.com
WHITELIST   REVDNS  .fedex.com
WHITELIST   REVDNS  .gartner.com
WHITELIST   REVDNS  .getactive.com
WHITELIST   REVDNS  .hertz.com
WHITELIST   REVDNS  .house.gov
WHITELIST   REVDNS  .ibm.com
WHITELIST   REVDNS  infoworld.wc09.net
WHITELIST   REVDNS  .ipswitch.com
WHITELIST   REVDNS  .j2.com
WHITELIST   REVDNS  .kintera.com
WHITELIST   REVDNS  .looksmart.com
WHITELIST   REVDNS  .luxurylink.com
WHITELIST   REVDNS  .macromedia.com
WHITELIST   REVDNS  .microsoft.com
WHITELIST   REVDNS  .microsoft.m0.net
WHITELIST   REVDNS  .moveon.org
WHITELIST   REVDNS  .msnbc.com
WHITELIST   REVDNS  .nytimes.com
WHITELIST   REVDNS  .officemax.com
WHITELIST   REVDNS  .openitx.com
WHITELIST   REVDNS  .oracle.com
WHITELIST   REVDNS  .paypal.com
WHITELIST   REVDNS  .philanthropy.com
WHITELIST   REVDNS  .schwab.com
WHITELIST   REVDNS  .sears.com
WHITELIST   REVDNS  .shockwave.com
WHITELIST   REVDNS  .thawte.com
WHITELIST   REVDNS  .travelzoo.com
WHITELIST   REVDNS  .truste.org
WHITELIST   REVDNS  .ups.com
WHITELIST   REVDNS  .usairways.com
WHITELIST   REVDNS  .veritas.com
WHITELIST   REVDNS  .zd-swx.com

Regards,
Kami 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Bill Landry
Sent: Sunday, September 14, 2003 10:39 PM
To: [EMAIL PROTECTED]
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


Kami, the only reason I mentioned PayPal to Matt was because I figured he
would be tracking FPs regarding his Obfuscation test.  The PayPal message in
question here did get delivered without user intervention, however, it was
not due to PayPal being whitelisted.

I don't like to whitelist anything except TO addresses, since anything
else that is whitelisted can be abused, including RDNS.  Instead, we apply a
high enough negative weight to three primary filter tests (HELO, RDNS 
MAILFROM) to trusted mailers so that they will generally pass with an
acceptable weight and get delivered without user intervention; however,
anything sent by a spammer abusing these trusted mailer addresses will still
likely get caught because they probably will not pass all three of these
primary tests, and will most likely fail other JunkMail tests, as well.

When something is whitelisted, no other tests can be run against these
messages and they simply get delivered, no matter what.  However, if you
instead apply a minimal negative weight to multiple tests, forged e-mail
will still likely get caught and not delivered.

Using PayPal as an example, if you whitelist RDNS, or MailFrom, or HELO,
etc., if a spammer happens to forge their messages using any of these, there
spam gets delivered, no matter what other tests it might have failed.
However, if you instead apply minimal negative weights like:

MAILFROM-5ENDSWITH.paypal.com
REVDNS-5ENDSWIDTH.paypal.com
HELO-5ENDSWITH.paypal.com

This give legitimate PayPal e-mail a total negative of -15, which will most
likely allow it to be delivered, even if it fail a couple of other tests.
However, the likelihood of a spammer being able to successfully meet all
three of these criteria is highly unlikely, and even if they did, there are
still all of the other spam tests that JunkMail supports that we can run
against these messages and still probably block it's delivery.  It basically
gives a fighting chance against forging spammers who attempt to abuse
spam-test whitelists.

Just my 2 cents...

Bill

- Original Message - 
From: Kami Razvan [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 6:04 PM
Subject: RE: [Declude.JunkMail] OBFUSCATION filter


 Bill

RE: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Keith Anderson

That was me, and thank you for posting that!

 Since someone asked about our whitelist- here it is (these 
 are the general
 items - we have in this list some of our clients with screwed 
 up server
 setups but are taken out in this list).  This goes in the 
 Global.cfg file.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Bill Landry
Kami, I hope there are no spammers monitoring this list since now they know
how to easily spam your e-mail domains.  It is never a good idea to share
your whitelists in a public forum.

Bill
- Original Message - 
From: Kami Razvan [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, September 15, 2003 4:42 AM
Subject: RE: [Declude.JunkMail] OBFUSCATION filter


Hi Bill:
You are right...  No disagreement here.

We had negative MAILFROM but it was being abused like crazy.  We were
getting so much spam from faked addresses.  We now have a negative list for
mailing lists and at times we see email coming through.

REVDNS whitelist has worked well and we have not yet seen any abuses - but
as a rule I agree with you it can be abused.

Since someone asked about our whitelist- here it is (these are the general
items - we have in this list some of our clients with screwed up server
setups but are taken out in this list).  This goes in the Global.cfg file.

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


RE: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Keith Anderson

Sorry, my fault for asking.

 Kami, I hope there are no spammers monitoring this list since 
 now they know
 how to easily spam your e-mail domains.  It is never a good 
 idea to share
 your whitelists in a public forum.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Jason Newland
But, Kami just listed the revdns whitelists, wouldn't the spammer have to
have a RDNS listing of something in her whitelist (not likely) to take
advantage of the listing?

Jason

- Original Message -
From: Keith Anderson [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, September 15, 2003 10:05 AM
Subject: RE: [Declude.JunkMail] OBFUSCATION filter



 Sorry, my fault for asking.

  Kami, I hope there are no spammers monitoring this list since
  now they know
  how to easily spam your e-mail domains.  It is never a good
  idea to share
  your whitelists in a public forum.


 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


RE: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Kami Razvan
Bill is right..

As a general rule it is not a good idea to post whitelists on a list.

REVDNS faking is not as easy as faking return email.. But as was discussed a
long time ago it is still possible.  Scott had a lengthy posting regarding
this indicating the difficulties but yet again it is possible.

It is a good practice to send those off list.

My mistake.. It has to be Monday again! ... I have not used my Monday's
quota for a long time so...

Regards,
Kami

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Jason Newland
Sent: Monday, September 15, 2003 11:21 AM
To: [EMAIL PROTECTED]
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


But, Kami just listed the revdns whitelists, wouldn't the spammer have to
have a RDNS listing of something in her whitelist (not likely) to take
advantage of the listing?

Jason

- Original Message -
From: Keith Anderson [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, September 15, 2003 10:05 AM
Subject: RE: [Declude.JunkMail] OBFUSCATION filter



 Sorry, my fault for asking.

  Kami, I hope there are no spammers monitoring this list since now 
  they know how to easily spam your e-mail domains.  It is never a 
  good idea to share
  your whitelists in a public forum.


 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Bill Landry
Yes, but since I run my own name servers, I could easily setup the IP
address of my mail server to respond to a reverse query with one of the
domains listed in his whitelist.  Granted, RDNS is more difficult to forge
then say HELO or MAILFROM, but is still fairly trivial if you run your own
name servers.

Bill
- Original Message - 
From: Jason Newland [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Monday, September 15, 2003 8:21 AM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 But, Kami just listed the revdns whitelists, wouldn't the spammer have to
 have a RDNS listing of something in her whitelist (not likely) to take
 advantage of the listing?

 Jason

 - Original Message -
 From: Keith Anderson [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Sent: Monday, September 15, 2003 10:05 AM
 Subject: RE: [Declude.JunkMail] OBFUSCATION filter


 
  Sorry, my fault for asking.
 
   Kami, I hope there are no spammers monitoring this list since
   now they know
   how to easily spam your e-mail domains.  It is never a good
   idea to share
   your whitelists in a public forum.
 
 
  ---
  [This E-mail was scanned for viruses by Declude Virus
 (http://www.declude.com)]
 
  ---
  This E-mail came from the Declude.JunkMail mailing list.  To
  unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
  type unsubscribe Declude.JunkMail.  The archives can be found
  at http://www.mail-archive.com.
 

 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread R. Scott Perry

Yes, but since I run my own name servers, I could easily setup the IP
address of my mail server to respond to a reverse query with one of the
domains listed in his whitelist.  Granted, RDNS is more difficult to forge
then say HELO or MAILFROM, but is still fairly trivial if you run your own
name servers.
Not only do you need your own nameservers, but you also need your upstream 
to delegate authority for the reverse DNS entries to you.  So any open 
relays or open proxies will not have forged reverse DNS.  Then, there are 
the potential legal consequences of a spammer using a reverse DNS entry 
like mail.paypal.com -- they could very likely get sued for trademark 
infringement, false advertising, etc.  And a spammer with the ability to 
change their own reverse DNS entries would be much easier to track down 
than a typical spammer.

So it definitely is possible, but unlikely.  I'm sure that if a spammer 
*does* change their reverse DNS entry to something that may commonly be 
whitelisted, it would be detected quite quickly (Gee, why did this spam 
get through -- ah, it was whitelisted, I wonder why? -- oh, the reverse DNS 
entry is mail.paypal.com).

   -Scott
---
Declude JunkMail: The advanced anti-spam solution for IMail mailservers.
Declude Virus: Catches known viruses and is the leader in mailserver 
vulnerability detection.
Find out what you've been missing: Ask about our free 30-day evaluation.

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Matthew Bramble
Pete,

It's not redundant because the two by themselves only check for strings 
of two, while the combination checks for strings with one of each in 
succession.  This way, if they go back and forth between the two, it 
will get caught as long as there is a . or @ between them, or as 
long as it is URL encoding followed by HTML encoding.  I left out the 
other way around because it was only a two character string, ;% and 
wanted to protect from FP's.

I do appreciate the feedback though...I do of course make mistakes.

Matt

Pete McNeil wrote:

Matt,

It appears that your coding for a combination of http  url encoding 
in urls is redundant since you capture both types individually. It's a 
small optimization, but worth mentioning.

_M

At 07:46 PM 9/14/2003 -0400, you wrote:

I've posted a newer version of the OBFUSCATION filter on my site.  
This contains the removal of the attachment thing and also the 
removal of 6 (of over 100) tests in order to be more forgiving, sans 
the PayPal issue.

http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003c.txt 

If you find any false positives with this besides the Ticketmaster 
one that I've already counterbalanced, please let me know.  I would 
imagine that posting to this group would be better than PM's unless 
others mind having discussion here.  That way everyone would know 
about any issues ASAP.

Thanks,

Matt

---
[This E-mail was scanned for viruses by Declude Virus 
(http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus 
(http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Bill Landry
- Original Message - 
From: R. Scott Perry [EMAIL PROTECTED]

 Not only do you need your own nameservers, but you also need your upstream
 to delegate authority for the reverse DNS entries to you.  So any open
 relays or open proxies will not have forged reverse DNS.  Then, there are
 the potential legal consequences of a spammer using a reverse DNS entry
 like mail.paypal.com -- they could very likely get sued for trademark
 infringement, false advertising, etc.  And a spammer with the ability to
 change their own reverse DNS entries would be much easier to track down
 than a typical spammer.

Yep, all of this it true, however, as a spammer I would only use the PTR for
that single spam run and then change it.  Spammers abuse trademarked names
in their HELO and MAILFROM addresses, why would you think they would be
opposed to using them in RDNS, if they have the ability to?  Again, my only
point was that it is not a good idea to share your whitelists on a public
forum, not the how-to's of spamming.

 So it definitely is possible, but unlikely.  I'm sure that if a spammer
 *does* change their reverse DNS entry to something that may commonly be
 whitelisted, it would be detected quite quickly (Gee, why did this spam
 get through -- ah, it was whitelisted, I wonder why? -- oh, the reverse
DNS
 entry is mail.paypal.com).

Still does not make it wise to share whitelists on a public forum.  However,
if you are promoting a whitelist exchange on this list, so be it; however,
it's not a practice I plan to participate in.

Bill

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


RE: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Pete - Madscientist
Ahh. Understood. I got confused by our rules where we code for a single
instance restricted to the URL. (Can't do that without wildcards). All
good then. Great work!
_M

|-Original Message-
|From: [EMAIL PROTECTED] 
|[mailto:[EMAIL PROTECTED] On Behalf Of 
|Matthew Bramble
|Sent: Monday, September 15, 2003 12:40 PM
|To: [EMAIL PROTECTED]
|Subject: Re: [Declude.JunkMail] OBFUSCATION filter
|
|
|Pete,
|
|It's not redundant because the two by themselves only check 
|for strings 
|of two, while the combination checks for strings with one of each in 
|succession.  This way, if they go back and forth between the two, it 
|will get caught as long as there is a . or @ between them, or as 
|long as it is URL encoding followed by HTML encoding.  I left out the 
|other way around because it was only a two character string, ;% and 
|wanted to protect from FP's.
|
|I do appreciate the feedback though...I do of course make mistakes.
|
|Matt
|
|Pete McNeil wrote:
|
| Matt,
|
| It appears that your coding for a combination of http  url encoding
| in urls is redundant since you capture both types 
|individually. It's a 
| small optimization, but worth mentioning.
|
| _M
|
| At 07:46 PM 9/14/2003 -0400, you wrote:
|
| I've posted a newer version of the OBFUSCATION filter on my site.
| This contains the removal of the attachment thing and also the 
| removal of 6 (of over 100) tests in order to be more 
|forgiving, sans 
| the PayPal issue.
|
| 
|http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003
| c.txt
|
|
| If you find any false positives with this besides the Ticketmaster
| one that I've already counterbalanced, please let me know.  I would 
| imagine that posting to this group would be better than PM's unless 
| others mind having discussion here.  That way everyone would know 
| about any issues ASAP.
|
| Thanks,
|
| Matt
|
| ---
| [This E-mail was scanned for viruses by Declude Virus
| (http://www.declude.com)]
|
| ---
| This E-mail came from the Declude.JunkMail mailing list.  To 
| unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type 
| unsubscribe Declude.JunkMail.  The archives can be found at 
| http://www.mail-archive.com.
|
|
| ---
| [This E-mail was scanned for viruses by Declude Virus
| (http://www.declude.com)]
|
| ---
| This E-mail came from the Declude.JunkMail mailing list.  To 
| unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type 
| unsubscribe Declude.JunkMail.  The archives can be found at 
| http://www.mail-archive.com.
|
|
|---
|[This E-mail was scanned for viruses by Declude Virus 
(http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type
unsubscribe Declude.JunkMail.  The archives can be found at
http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Matthew Bramble
Bill Landry wrote:

Still does not make it wise to share whitelists on a public forum.  However,
if you are promoting a whitelist exchange on this list, so be it; however,
it's not a practice I plan to participate in.
I have less than 500 addresses being used on my server and only about 
250 accounts.  If spammers want to customize their attack for my 
vunerabilities...I would consider that to be an honor and a waste of 
their resources, and therefore a net good.  Of course they won't 
though...not for me at least.

On the other hand, if I was working for AOL and posting their 
whitelist...that would be a whole 'nother matter.

Matt



---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Bill Landry
- Original Message - 
From: Matthew Bramble [EMAIL PROTECTED]

 Still does not make it wise to share whitelists on a public forum.
However,
 if you are promoting a whitelist exchange on this list, so be it;
however,
 it's not a practice I plan to participate in.
 

 I have less than 500 addresses being used on my server and only about
 250 accounts.  If spammers want to customize their attack for my
 vunerabilities...I would consider that to be an honor and a waste of
 their resources, and therefore a net good.  Of course they won't
 though...not for me at least.

 On the other hand, if I was working for AOL and posting their
 whitelist...that would be a whole 'nother matter.

Hmmm, you seem to be missing the point.  Spammers monitor these spam lists
in order to learn how to subvert spam filters, so why make there jobs any
easier and your user any more vulnerable?

Bill

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Matthew Bramble
Bill Landry wrote:

Hmmm, you seem to be missing the point.  Spammers monitor these spam lists
in order to learn how to subvert spam filters, so why make there jobs any
easier and your user any more vulnerable?
None of this stuff is a big secret, and besides, pretending to come from 
a domain like AOL or Amazon has resulted in spammers being sued 
successfully.  Clearly they already know the tactics and have used them.

On the other hand, if I wanted to become a spammer, I assure you that I 
could get past your spam filters with near perfect success.  Most of 
these guys don't even know how to fake a header properly and that would 
take someone moderately intelligent about 5 seconds to figure out.  It's 
the fact that these guys are so dumb that makes it so that we can block 
them as effectively as we do.  In the future, the only way around this 
will a distributed network of truly real-time, reliable blocklists where 
trusted people are promoting spam instead of spamtraps.  Spamcop is 
doing this to some extent, but they lack in quality control because of 
the automation and lack of attention to whitelisting.  They blocked 
PayPal the other day for at least several hours for instance...that got 
them demoted on my server.  Same goes for MailPolice, who somehow tagged 
Ebay as porn.

Matt



---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Bill Landry
- Original Message - 
From: Matthew Bramble [EMAIL PROTECTED]


 None of this stuff is a big secret, and besides, pretending to come from
 a domain like AOL or Amazon has resulted in spammers being sued
 successfully.  Clearly they already know the tactics and have used them.

And these successful lawsuits have obviously not stopped the practice.

 On the other hand, if I wanted to become a spammer, I assure you that I
 could get past your spam filters with near perfect success.

Although I highly doubt it, your point is...?

  Most of
 these guys don't even know how to fake a header properly and that would
 take someone moderately intelligent about 5 seconds to figure out.  It's
 the fact that these guys are so dumb that makes it so that we can block
 them as effectively as we do.

So let's make it easier for them by posting our whitelists.  This is
straying all over the place.  If you think it is fine and good to post your
whitelists on a public forum, then by all means do so.  It's was just my
personal recommendation that it is not a wise thing to do, but to each his
own...

Bill

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


RE: [Declude.JunkMail] OBFUSCATION filter

2003-09-15 Thread Kevin Bilbee
I know this is a little late to the party. But I do think Spammers monitor
this list. A few weeks back I posted some IP addresses that I was receiving
spam from. I have not recieved a single spam from thoes servers since but
other users/domains on my server have.

I have them spamtraped so I can monitor the volume.

Not a good Idea to post whitelists to and spamfiltering user list.


Kevin Bilbee

 -Original Message-
 From: [EMAIL PROTECTED]
 [mailto:[EMAIL PROTECTED] Behalf Of Kami Razvan
 Sent: Monday, September 15, 2003 4:42 AM
 To: [EMAIL PROTECTED]
 Subject: RE: [Declude.JunkMail] OBFUSCATION filter


 Hi Bill:
 You are right...  No disagreement here.

 We had negative MAILFROM but it was being abused like crazy.  We were
 getting so much spam from faked addresses.  We now have a
 negative list for
 mailing lists and at times we see email coming through.

 REVDNS whitelist has worked well and we have not yet seen any abuses - but
 as a rule I agree with you it can be abused.

 Since someone asked about our whitelist- here it is (these are the general
 items - we have in this list some of our clients with screwed up server
 setups but are taken out in this list).  This goes in the Global.cfg file.

 WHITELIST REVDNS  .airborne.com
 WHITELIST REVDNS  .amazon.com
 WHITELIST REVDNS  .audible.com
 WHITELIST REVDNS  .bestfares.com
 WHITELIST REVDNS  .cnet.com
 WHITELIST REVDNS  .dell.com
 WHITELIST REVDNS  .dowjones.com
 WHITELIST REVDNS  .ebay.com
 WHITELIST REVDNS  .equifax.com
 WHITELIST REVDNS  .fedex.com
 WHITELIST REVDNS  .gartner.com
 WHITELIST REVDNS  .getactive.com
 WHITELIST REVDNS  .hertz.com
 WHITELIST REVDNS  .house.gov
 WHITELIST REVDNS  .ibm.com
 WHITELIST REVDNS  infoworld.wc09.net
 WHITELIST REVDNS  .ipswitch.com
 WHITELIST REVDNS  .j2.com
 WHITELIST REVDNS  .kintera.com
 WHITELIST REVDNS  .looksmart.com
 WHITELIST REVDNS  .luxurylink.com
 WHITELIST REVDNS  .macromedia.com
 WHITELIST REVDNS  .microsoft.com
 WHITELIST REVDNS  .microsoft.m0.net
 WHITELIST REVDNS  .moveon.org
 WHITELIST REVDNS  .msnbc.com
 WHITELIST REVDNS  .nytimes.com
 WHITELIST REVDNS  .officemax.com
 WHITELIST REVDNS  .openitx.com
 WHITELIST REVDNS  .oracle.com
 WHITELIST REVDNS  .paypal.com
 WHITELIST REVDNS  .philanthropy.com
 WHITELIST REVDNS  .schwab.com
 WHITELIST REVDNS  .sears.com
 WHITELIST REVDNS  .shockwave.com
 WHITELIST REVDNS  .thawte.com
 WHITELIST REVDNS  .travelzoo.com
 WHITELIST REVDNS  .truste.org
 WHITELIST REVDNS  .ups.com
 WHITELIST REVDNS  .usairways.com
 WHITELIST REVDNS  .veritas.com
 WHITELIST REVDNS  .zd-swx.com

 Regards,
 Kami

 -Original Message-
 From: [EMAIL PROTECTED]
 [mailto:[EMAIL PROTECTED] On Behalf Of Bill Landry
 Sent: Sunday, September 14, 2003 10:39 PM
 To: [EMAIL PROTECTED]
 Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 Kami, the only reason I mentioned PayPal to Matt was because I figured he
 would be tracking FPs regarding his Obfuscation test.  The PayPal
 message in
 question here did get delivered without user intervention, however, it was
 not due to PayPal being whitelisted.

 I don't like to whitelist anything except TO addresses, since anything
 else that is whitelisted can be abused, including RDNS.  Instead,
 we apply a
 high enough negative weight to three primary filter tests (HELO, RDNS 
 MAILFROM) to trusted mailers so that they will generally pass with an
 acceptable weight and get delivered without user intervention; however,
 anything sent by a spammer abusing these trusted mailer addresses
 will still
 likely get caught because they probably will not pass all three of these
 primary tests, and will most likely fail other JunkMail tests, as well.

 When something is whitelisted, no other tests can be run against these
 messages and they simply get delivered, no matter what.  However, if you
 instead apply a minimal negative weight to multiple tests, forged e-mail
 will still likely get caught and not delivered.

 Using PayPal as an example, if you whitelist RDNS, or MailFrom, or HELO,
 etc., if a spammer happens to forge their messages using any of
 these, there
 spam gets delivered, no matter what other tests it might have failed.
 However, if you instead apply minimal negative weights like:

 MAILFROM-5ENDSWITH.paypal.com
 REVDNS-5ENDSWIDTH.paypal.com
 HELO-5ENDSWITH.paypal.com

 This give legitimate PayPal e-mail a total negative of -15, which
 will most
 likely allow it to be delivered, even if it fail

[Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Matthew Bramble
I put together a filter that checks for obfuscation of URL's, IP's and 
text using URL encoding, HTML encoding, a mix of URL and HTML encoding, 
Hexadecimal encoding, and octal encoding, though the latter two are 
commented out due to a lack of current use by spammers.  I've been 
careful to allow hits only on combinations of either letters and numbers 
or letters and numbers with HTTP address components in order to protect 
from false positives.  The technique is probably about the most 
foolproof non-specific indicative indicator of spam that there is, and 
should prove to be more reliable than most any other test out there.

My results from a smattering of E-mail tested with this filter are as 
follows:

   805 - Unique Messages
34 - Filter Hits (4.2%)
 0 - False Positives
 4 - Made a difference (would have scored within 50% of my fail 
weight without the test)
 3 - Failed because of the test.

I'm going to attach the file to a separate posting just in case some 
people are already filtering for these techniques.  I might suggest 
trying not to include the text of the filter in replies, especially in 
PM's direct to my account :)

Special credit goes to Dan for leading me in the direction of obfuscation.

Matt

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


[Declude.JunkMail] OBFUSCATION filter - attachment

2003-09-14 Thread Matthew Bramble
This is the obfuscation filter attached.  Please don't reply to the 
other message or don't include the filter.

Matt
# OBFUSCATION
# Last Update: 09/14/2003
#
# Description:
# Encoding of letters and numbers in E-mail is unnecessary, however various techniques 
are
# sometimes used by spammers to hide from filters, even mixing multiple techniques in 
URL's at
# times.  This filter will detect text and URL encoding only in combinations where 
multiple
# encoded numbers and characters are in succession or mixed with HTTP address 
components.  More
# information on URL obfuscation techniques can be found at: 
http://www.pc-help.org/obscure.htm
#
# Usage:
# OBFUSCATION filter C:\IMail\Declude\Obfuscation.txt x 7 0
#
# False Positives:
# Web designers and programmers passing code, ASCII text art, and legitimate bulk 
mailers that
# needlessly URL encode letters and numbers in their script arguments (only special 
characters
# are necessary).  False positives are extremely rare.


# Counterbalances:
# Negative weighting is applied for responsible bulk mailers that fail this test. In 
order to futhur
# protect from the possibility of HTML or scripting file attachments triggering this 
filter, an
# optional counterbalance for all E-mail with attachments can be used, however using 
it would mark
# all E-mail with attachments, however it would not score them.
#
# Test Exclusions:
# Attachments, and Ticketmaster.

#BODY   -7  CONTAINScontent-disposition: attachment
MAILFROM-7  ENDSWITHticketmaster.com


# URL Encoded Obfuscation:
# This technique is used to obfuscate URL's.  The filter will only match two 
characters in
# succession with the first being a letter or number in order to protect form false 
positives.
#
# Example:
# http://%77%77%77.%67%6F%6F%67%6C%65.%63%6F%6D/

# 0-9

BODY0   CONTAINS%30%
BODY0   CONTAINS%31%
BODY0   CONTAINS%32%
BODY0   CONTAINS%33%
BODY0   CONTAINS%34%
BODY0   CONTAINS%35%
BODY0   CONTAINS%36%
BODY0   CONTAINS%37%
BODY0   CONTAINS%38%
BODY0   CONTAINS%39%

# A-Z

BODY0   CONTAINS%41%
BODY0   CONTAINS%42%
BODY0   CONTAINS%43%
BODY0   CONTAINS%44%
BODY0   CONTAINS%45%
BODY0   CONTAINS%46%
BODY0   CONTAINS%47%
BODY0   CONTAINS%48%
BODY0   CONTAINS%49%
BODY0   CONTAINS%4a%
BODY0   CONTAINS%4b%
BODY0   CONTAINS%4c%
BODY0   CONTAINS%4d%
BODY0   CONTAINS%4e%
BODY0   CONTAINS%4f%
BODY0   CONTAINS%50%
BODY0   CONTAINS%51%
BODY0   CONTAINS%52%
BODY0   CONTAINS%53%
BODY0   CONTAINS%54%
BODY0   CONTAINS%55%
BODY0   CONTAINS%56%
BODY0   CONTAINS%57%
BODY0   CONTAINS%58%
BODY0   CONTAINS%59%
BODY0   CONTAINS%5a%

# a-z

BODY0   CONTAINS%61%
BODY0   CONTAINS%62%
BODY0   CONTAINS%63%
BODY0   CONTAINS%64%
BODY0   CONTAINS%65%
BODY0   CONTAINS%66%
BODY0   CONTAINS%67%
BODY0   CONTAINS%68%
BODY0   CONTAINS%69%
BODY0   CONTAINS%6a%
BODY0   CONTAINS%6b%
BODY0   CONTAINS%6c%
BODY0   CONTAINS%6d%
BODY0   CONTAINS%6e%
BODY0   CONTAINS%6f%
BODY0   CONTAINS%70%
BODY0   CONTAINS%71%
BODY0   CONTAINS%72%
BODY0   CONTAINS%73%
BODY0   CONTAINS%74%
BODY0   CONTAINS%75%
BODY0   CONTAINS%76%
BODY0   CONTAINS%77%
BODY0   CONTAINS%78%
BODY0   CONTAINS%79%
BODY0   CONTAINS%7a%

# With HTTP

BODY0   CONTAINShttp://%
BODY0   CONTAINS[EMAIL PROTECTED]
BODY0   CONTAINS%.%


# HTML Encoded Obfuscation:
# This technique is used to obfuscate URL's and hide keywords.  The filter will only 
match
# two characters in succession with the first being a letter or number in order to 
protect
# form false positives.
#
# Examples:
# A 
HREF=http://#119;#119;#119;.#103;#111;#111;#103;#108;#101;.#99;#111;#109;/;Google/A
# V#73;AG#82;A

# 0-9

BODY0   CONTAINS#48;#
BODY0   CONTAINS#49;#
BODY0   CONTAINS#50;#
BODY0   CONTAINS#51;#
BODY0   CONTAINS#52;#
BODY0   CONTAINS#53;#
BODY0   

Re: [Declude.JunkMail] OBFUSCATION filter - attachment - replacement!

2003-09-14 Thread Matthew Bramble
I just figured out that the attachment exclusion thing doesn't work as 
desired so I removed everything pertaining to that (oops).  The chance 
of a false positive occurring are very low even without the ability to 
exclude inline attachments that might contain raw scripting or HTML code.

Please use this updated file instead if you want to test out the 
filter.  Also please post any FP's that you believe should be 
counterbalanced in the test like the Ticketmaster example.

Thanks,

Matt
# OBFUSCATION
# Last Update: 09/14/2003
#
# Description:
# Encoding of letters and numbers in E-mail is unnecessary, however various techniques 
are
# sometimes used by spammers to hide from filters, even mixing multiple techniques in 
URL's at
# times.  This filter will detect text and URL encoding only in combinations where 
multiple
# encoded numbers and characters are in succession or mixed with HTTP address 
components.  More
# information on URL obfuscation techniques can be found at: 
http://www.pc-help.org/obscure.htm
#
# Usage:
# OBFUSCATION filter C:\IMail\Declude\Obfuscation.txt x 7 0
#
# False Positives:
# Web designers and programmers passing inline code, ASCII text art, and legitimate 
bulk mailers
# that needlessly URL encode letters and numbers in their script arguments (only 
special
# characters are necessary).  False positives are extremely rare.


# Counterbalances:
# Negative weighting is applied for responsible bulk mailers that fail this test.
#
# Test Exclusions:
# Ticketmaster.

MAILFROM-7  ENDSWITHticketmaster.com


# URL Encoded Obfuscation:
# This technique is used to obfuscate URL's.  The filter will only match two 
characters in
# succession with the first being a letter or number in order to protect form false 
positives.
#
# Example:
# http://%77%77%77.%67%6F%6F%67%6C%65.%63%6F%6D/

# 0-9

BODY0   CONTAINS%30%
BODY0   CONTAINS%31%
BODY0   CONTAINS%32%
BODY0   CONTAINS%33%
BODY0   CONTAINS%34%
BODY0   CONTAINS%35%
BODY0   CONTAINS%36%
BODY0   CONTAINS%37%
BODY0   CONTAINS%38%
BODY0   CONTAINS%39%

# A-Z

BODY0   CONTAINS%41%
BODY0   CONTAINS%42%
BODY0   CONTAINS%43%
BODY0   CONTAINS%44%
BODY0   CONTAINS%45%
BODY0   CONTAINS%46%
BODY0   CONTAINS%47%
BODY0   CONTAINS%48%
BODY0   CONTAINS%49%
BODY0   CONTAINS%4a%
BODY0   CONTAINS%4b%
BODY0   CONTAINS%4c%
BODY0   CONTAINS%4d%
BODY0   CONTAINS%4e%
BODY0   CONTAINS%4f%
BODY0   CONTAINS%50%
BODY0   CONTAINS%51%
BODY0   CONTAINS%52%
BODY0   CONTAINS%53%
BODY0   CONTAINS%54%
BODY0   CONTAINS%55%
BODY0   CONTAINS%56%
BODY0   CONTAINS%57%
BODY0   CONTAINS%58%
BODY0   CONTAINS%59%
BODY0   CONTAINS%5a%

# a-z

BODY0   CONTAINS%61%
BODY0   CONTAINS%62%
BODY0   CONTAINS%63%
BODY0   CONTAINS%64%
BODY0   CONTAINS%65%
BODY0   CONTAINS%66%
BODY0   CONTAINS%67%
BODY0   CONTAINS%68%
BODY0   CONTAINS%69%
BODY0   CONTAINS%6a%
BODY0   CONTAINS%6b%
BODY0   CONTAINS%6c%
BODY0   CONTAINS%6d%
BODY0   CONTAINS%6e%
BODY0   CONTAINS%6f%
BODY0   CONTAINS%70%
BODY0   CONTAINS%71%
BODY0   CONTAINS%72%
BODY0   CONTAINS%73%
BODY0   CONTAINS%74%
BODY0   CONTAINS%75%
BODY0   CONTAINS%76%
BODY0   CONTAINS%77%
BODY0   CONTAINS%78%
BODY0   CONTAINS%79%
BODY0   CONTAINS%7a%

# With HTTP

BODY0   CONTAINShttp://%
BODY0   CONTAINS[EMAIL PROTECTED]
BODY0   CONTAINS%.%


# HTML Encoded Obfuscation:
# This technique is used to obfuscate URL's and hide keywords.  The filter will only 
match
# two characters in succession with the first being a letter or number in order to 
protect
# form false positives.
#
# Examples:
# A 
HREF=http://#119;#119;#119;.#103;#111;#111;#103;#108;#101;.#99;#111;#109;/;Google/A
# V#73;AG#82;A

# 0-9

BODY0   CONTAINS#48;#
BODY0   CONTAINS#49;#
BODY0   CONTAINS#50;#
BODY0   CONTAINS#51;#
BODY0   CONTAINS#52;#
BODY0   CONTAINS#53;#
BODY   

Re: [Declude.JunkMail] OBFUSCATION filter - attachment - replacement!

2003-09-14 Thread Bill Landry
Very nice work, Matt!  And thanks a bunch for sharing your efforts with the
list!

Bill
- Original Message - 
From: Matthew Bramble [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 10:14 AM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter - attachment -
replacement!


 I just figured out that the attachment exclusion thing doesn't work as
 desired so I removed everything pertaining to that (oops).  The chance
 of a false positive occurring are very low even without the ability to
 exclude inline attachments that might contain raw scripting or HTML code.

 Please use this updated file instead if you want to test out the
 filter.  Also please post any FP's that you believe should be
 counterbalanced in the test like the Ticketmaster example.

 Thanks,

 Matt








 # OBFUSCATION
 # Last Update: 09/14/2003
 #
 # Description:
 # Encoding of letters and numbers in E-mail is unnecessary, however
various techniques are
 # sometimes used by spammers to hide from filters, even mixing multiple
techniques in URL's at
 # times.  This filter will detect text and URL encoding only in
combinations where multiple
 # encoded numbers and characters are in succession or mixed with HTTP
address components.  More
 # information on URL obfuscation techniques can be found at:
http://www.pc-help.org/obscure.htm
 #
 # Usage:
 # OBFUSCATION filter C:\IMail\Declude\Obfuscation.txt x 7
0
 #
 # False Positives:
 # Web designers and programmers passing inline code, ASCII text art, and
legitimate bulk mailers
 # that needlessly URL encode letters and numbers in their script arguments
(only special
 # characters are necessary).  False positives are extremely rare.


 # Counterbalances:
 # Negative weighting is applied for responsible bulk mailers that fail
this test.
 #
 # Test Exclusions:
 # Ticketmaster.

 MAILFROM -7 ENDSWITH ticketmaster.com


 # URL Encoded Obfuscation:
 # This technique is used to obfuscate URL's.  The filter will only match
two characters in
 # succession with the first being a letter or number in order to protect
form false positives.
 #
 # Example:
 # http://%77%77%77.%67%6F%6F%67%6C%65.%63%6F%6D/

 # 0-9

 BODY 0 CONTAINS %30%
 BODY 0 CONTAINS %31%
 BODY 0 CONTAINS %32%
 BODY 0 CONTAINS %33%
 BODY 0 CONTAINS %34%
 BODY 0 CONTAINS %35%
 BODY 0 CONTAINS %36%
 BODY 0 CONTAINS %37%
 BODY 0 CONTAINS %38%
 BODY 0 CONTAINS %39%

 # A-Z

 BODY 0 CONTAINS %41%
 BODY 0 CONTAINS %42%
 BODY 0 CONTAINS %43%
 BODY 0 CONTAINS %44%
 BODY 0 CONTAINS %45%
 BODY 0 CONTAINS %46%
 BODY 0 CONTAINS %47%
 BODY 0 CONTAINS %48%
 BODY 0 CONTAINS %49%
 BODY 0 CONTAINS %4a%
 BODY 0 CONTAINS %4b%
 BODY 0 CONTAINS %4c%
 BODY 0 CONTAINS %4d%
 BODY 0 CONTAINS %4e%
 BODY 0 CONTAINS %4f%
 BODY 0 CONTAINS %50%
 BODY 0 CONTAINS %51%
 BODY 0 CONTAINS %52%
 BODY 0 CONTAINS %53%
 BODY 0 CONTAINS %54%
 BODY 0 CONTAINS %55%
 BODY 0 CONTAINS %56%
 BODY 0 CONTAINS %57%
 BODY 0 CONTAINS %58%
 BODY 0 CONTAINS %59%
 BODY 0 CONTAINS %5a%

 # a-z

 BODY 0 CONTAINS %61%
 BODY 0 CONTAINS %62%
 BODY 0 CONTAINS %63%
 BODY 0 CONTAINS %64%
 BODY 0 CONTAINS %65%
 BODY 0 CONTAINS %66%
 BODY 0 CONTAINS %67%
 BODY 0 CONTAINS %68%
 BODY 0 CONTAINS %69%
 BODY 0 CONTAINS %6a%
 BODY 0 CONTAINS %6b%
 BODY 0 CONTAINS %6c%
 BODY 0 CONTAINS %6d%
 BODY 0 CONTAINS %6e%
 BODY 0 CONTAINS %6f%
 BODY 0 CONTAINS %70%
 BODY 0 CONTAINS %71%
 BODY 0 CONTAINS %72%
 BODY 0 CONTAINS %73%
 BODY 0 CONTAINS %74%
 BODY 0 CONTAINS %75%
 BODY 0 CONTAINS %76%
 BODY 0 CONTAINS %77%
 BODY 0 CONTAINS %78%
 BODY 0 CONTAINS %79%
 BODY 0 CONTAINS %7a%

 # With HTTP

 BODY 0 CONTAINS http://%
 BODY 0 CONTAINS [EMAIL PROTECTED]
 BODY 0 CONTAINS %.%


 # HTML Encoded Obfuscation:
 # This technique is used to obfuscate URL's and hide keywords.  The filter
will only match
 # two characters in succession with the first being a letter or number in
order to protect
 # form false positives.
 #
 # Examples:
 # A
HREF=http://#119;#119;#119;.#103;#111;#111;#103;#108;#101;.#99;#
111;#109;/Google/A
 # V#73;AG#82;A

 # 0-9

 BODY 0 CONTAINS #48;#
 BODY 0 CONTAINS #49;#
 BODY 0 CONTAINS #50;#
 BODY 0 CONTAINS #51;#
 BODY 0 CONTAINS #52;#
 BODY 0 CONTAINS #53;#
 BODY 0 CONTAINS #54;#
 BODY 0 CONTAINS #55;#
 BODY 0 CONTAINS #56;#
 BODY 0 CONTAINS #57;#

 # A-Z

 BODY 0 CONTAINS #65;#
 BODY 0 CONTAINS #66;#
 BODY 0 CONTAINS #67;#
 BODY 0 CONTAINS #68;#
 BODY 0 CONTAINS #69;#
 BODY 0 CONTAINS #70;#
 BODY 0 CONTAINS #71;#
 BODY 0 CONTAINS #72;#
 BODY 0 CONTAINS #73;#
 BODY 0 CONTAINS #74;#
 BODY 0 CONTAINS #75;#
 BODY 0 CONTAINS #76;#
 BODY 0 CONTAINS #77;#
 BODY 0 CONTAINS #78;#
 BODY 0 CONTAINS #79;#
 BODY 0 CONTAINS #80;#
 BODY 0 CONTAINS #81;#
 BODY 0 CONTAINS #82;#
 BODY 0 CONTAINS #83;#
 BODY 0 CONTAINS #84;#
 BODY 0 CONTAINS #85;#
 BODY 0 CONTAINS #86;#
 BODY 0 CONTAINS #87;#
 BODY 0 CONTAINS #88;#
 BODY 0 CONTAINS #89;#
 BODY 0 CONTAINS #90;#

 # a-z

 BODY 0 CONTAINS #97;#
 BODY 0

Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Matthew Bramble
Thanks Bill.  And I've got a few more in me I believe :)

Matt

Bill Landry wrote:

Very nice work, Matt!  And thanks a bunch for sharing your efforts with the
list!
Bill
 

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Bill Landry
Just an FYI, I've added:

MAILFROM -7 ENDSWITH paypal.com

to the Test Exclusions, as it was flagged by the Obfuscation test.

Bill
- Original Message - 
From: Matthew Bramble [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 12:27 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 Thanks Bill.  And I've got a few more in me I believe :)
 
 Matt

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Frederick Samarelli
Would please share this filter.

Thanks
- Original Message - 
From: Matthew Bramble [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 12:28 PM
Subject: [Declude.JunkMail] OBFUSCATION filter


 I put together a filter that checks for obfuscation of URL's, IP's and
 text using URL encoding, HTML encoding, a mix of URL and HTML encoding,
 Hexadecimal encoding, and octal encoding, though the latter two are
 commented out due to a lack of current use by spammers.  I've been
 careful to allow hits only on combinations of either letters and numbers
 or letters and numbers with HTTP address components in order to protect
 from false positives.  The technique is probably about the most
 foolproof non-specific indicative indicator of spam that there is, and
 should prove to be more reliable than most any other test out there.

 My results from a smattering of E-mail tested with this filter are as
 follows:

 805 - Unique Messages
  34 - Filter Hits (4.2%)
   0 - False Positives
   4 - Made a difference (would have scored within 50% of my fail
 weight without the test)
   3 - Failed because of the test.

 I'm going to attach the file to a separate posting just in case some
 people are already filtering for these techniques.  I might suggest
 trying not to include the text of the filter in replies, especially in
 PM's direct to my account :)

 Special credit goes to Dan for leading me in the direction of obfuscation.

 Matt

 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Bill Landry
He did share it with the list--possibly your filters blocked the message.
If you are not automatically deleting messages, check you hold queue, you
may find it there.

Bill
- Original Message - 
From: Frederick Samarelli [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 12:49 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 Would please share this filter.

 Thanks
 - Original Message - 
 From: Matthew Bramble [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Sent: Sunday, September 14, 2003 12:28 PM
 Subject: [Declude.JunkMail] OBFUSCATION filter


  I put together a filter that checks for obfuscation of URL's, IP's and
  text using URL encoding, HTML encoding, a mix of URL and HTML encoding,
  Hexadecimal encoding, and octal encoding, though the latter two are
  commented out due to a lack of current use by spammers.  I've been
  careful to allow hits only on combinations of either letters and numbers
  or letters and numbers with HTTP address components in order to protect
  from false positives.  The technique is probably about the most
  foolproof non-specific indicative indicator of spam that there is, and
  should prove to be more reliable than most any other test out there.
 
  My results from a smattering of E-mail tested with this filter are as
  follows:
 
  805 - Unique Messages
   34 - Filter Hits (4.2%)
0 - False Positives
4 - Made a difference (would have scored within 50% of my fail
  weight without the test)
3 - Failed because of the test.
 
  I'm going to attach the file to a separate posting just in case some
  people are already filtering for these techniques.  I might suggest
  trying not to include the text of the filter in replies, especially in
  PM's direct to my account :)
 
  Special credit goes to Dan for leading me in the direction of
obfuscation.
 
  Matt
 
  ---
  [This E-mail was scanned for viruses by Declude Virus
 (http://www.declude.com)]
 
  ---
  This E-mail came from the Declude.JunkMail mailing list.  To
  unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
  type unsubscribe Declude.JunkMail.  The archives can be found
  at http://www.mail-archive.com.
 

 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Matthew Bramble




Bill (and others), if you find an exclusion, would it be possible to
post the offending code to the list? This way I can keep track of the
types of things FP'ing with the test and remove code that isn't
necessarily useful.

I think I found the PayPal issue, seems that they have a program that
isn't filling in a variable in this tag:

 img src="" class="moz-txt-link-rfc2396E" href="http://%3%/images/pixel.gif">"http://%3%/images/pixel.gif" height="5" width="1"
border="0"

It's a broken tag instead of them trying to obfuscate the address.
It's hitting the following line in that filter:

 BODY  0 CONTAINS http://%

The tests with HTTP stuff could be overkill and it opens the
possibility of hitting http:// followed by a non-number or letter.
Maybe I should remove those test instead of removing PayPal? I doubt
the test will be measurably weaker without the lines that do this type
of thing. I think that's what I'll do just to be safe.

In a day or so, I'll just start putting these in a folder on my site
instead of posting them to the list so that people's filters don't
start hitting them. Of course I would recommend whitelisting either
the group, or better yet Declude in the subject, that way PM's from the
list don't get blocked.

Matt



Bill Landry wrote:

  Just an FYI, I've added:

MAILFROM -7 ENDSWITH paypal.com

to the "Test Exclusions", as it was flagged by the Obfuscation test.

Bill
- Original Message - 
From: "Matthew Bramble" [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 12:27 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


  
  
Thanks Bill.  And I've got a few more in me I believe :)

Matt

  
  
---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail".  The archives can be found
at http://www.mail-archive.com.


  


-- 
===
Matthew S. Bramble
President and Technical Coordinator
iGaia Incorporated, Operator of NYcars.com
---
Office Phone: (518) 862-9042
Cellular: (518) 229-3375
Fax: (518) 862-9044
E-mail: [EMAIL PROTECTED] or [EMAIL PROTECTED]
===




Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Frederick Samarelli
You were correct.

Thanks.


- Original Message - 
From: Bill Landry [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 4:08 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 He did share it with the list--possibly your filters blocked the message.
 If you are not automatically deleting messages, check you hold queue, you
 may find it there.

 Bill
 - Original Message - 
 From: Frederick Samarelli [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Sent: Sunday, September 14, 2003 12:49 PM
 Subject: Re: [Declude.JunkMail] OBFUSCATION filter


  Would please share this filter.
 
  Thanks
  - Original Message - 
  From: Matthew Bramble [EMAIL PROTECTED]
  To: [EMAIL PROTECTED]
  Sent: Sunday, September 14, 2003 12:28 PM
  Subject: [Declude.JunkMail] OBFUSCATION filter
 
 
   I put together a filter that checks for obfuscation of URL's, IP's and
   text using URL encoding, HTML encoding, a mix of URL and HTML
encoding,
   Hexadecimal encoding, and octal encoding, though the latter two are
   commented out due to a lack of current use by spammers.  I've been
   careful to allow hits only on combinations of either letters and
numbers
   or letters and numbers with HTTP address components in order to
protect
   from false positives.  The technique is probably about the most
   foolproof non-specific indicative indicator of spam that there is, and
   should prove to be more reliable than most any other test out there.
  
   My results from a smattering of E-mail tested with this filter are as
   follows:
  
   805 - Unique Messages
34 - Filter Hits (4.2%)
 0 - False Positives
 4 - Made a difference (would have scored within 50% of my fail
   weight without the test)
 3 - Failed because of the test.
  
   I'm going to attach the file to a separate posting just in case some
   people are already filtering for these techniques.  I might suggest
   trying not to include the text of the filter in replies, especially in
   PM's direct to my account :)
  
   Special credit goes to Dan for leading me in the direction of
 obfuscation.
  
   Matt
  
   ---
   [This E-mail was scanned for viruses by Declude Virus
  (http://www.declude.com)]
  
   ---
   This E-mail came from the Declude.JunkMail mailing list.  To
   unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
   type unsubscribe Declude.JunkMail.  The archives can be found
   at http://www.mail-archive.com.
  
 
  ---
  [This E-mail was scanned for viruses by Declude Virus
 (http://www.declude.com)]
 
  ---
  This E-mail came from the Declude.JunkMail mailing list.  To
  unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
  type unsubscribe Declude.JunkMail.  The archives can be found
  at http://www.mail-archive.com.
 

 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Bill Landry



Good point, and yes it was the:

 BODY  0 
CONTAINS http://%
entry that flagged the PayPal message.

Bill

  - Original Message - 
  From: 
  Matthew Bramble 

  To: [EMAIL PROTECTED] 
  
  Sent: Sunday, September 14, 2003 1:17 
  PM
  Subject: Re: [Declude.JunkMail] 
  OBFUSCATION filter
  Bill (and others), if you find an exclusion, would it be 
  possible to post the offending code to the list? This way I can keep 
  track of the types of things FP'ing with the test and remove code that isn't 
  necessarily useful.


RE: [Declude.JunkMail] OBFUSCATION filter - attachment

2003-09-14 Thread Keith Anderson

Great work, Matt.

Is anyone aware of a repository web page out there with a collection of
Declude related things like this?  If not, someone ought to start one.  I'm
willing to do so if it doesn't already exist.

Regards,
Keith

 -Original Message-
 From: Matthew Bramble [mailto:[EMAIL PROTECTED]
 Sent: Sunday, September 14, 2003 10:31 AM
 To: [EMAIL PROTECTED]
 Subject: [Declude.JunkMail] OBFUSCATION filter - attachment


 This is the obfuscation filter attached.  Please don't reply to the
 other message or don't include the filter.

 Matt



---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter - attachment

2003-09-14 Thread Matthew Bramble
One obvious choice would be the tools section of the Declude site, but 
I'm already in the process of putting up a site for these filters, and I 
would be happy to give space to anyone else that wants to share.  DNS 
for this domain should be active tomorrow morning with some basic Web 
pages and downloads.

One of the reasons why I've been diving into this stuff so deeply is 
that I intend on writing some add-ons for Declude and/or IMail (blocking 
spam is of course the other one).  I've got about 4 ideas for things 
that would add some important functionality but seem outside of the 
scope of what's out there already.  One idea is in a functional spec 
already and we're just waiting on some information from Ipswitch before 
we start.  I would imagine that we are several weeks away from having an 
alpha version, and I will be sure to share when it's beta.

Let me know if anyone is interested in also listing things on this 
site.  Personally I've been intending on getting SPAMDOMAINS up and 
running, but finding information on these things is a bit convoluted at 
present.

Matt



Keith Anderson wrote:

Great work, Matt.

Is anyone aware of a repository web page out there with a collection of
Declude related things like this?  If not, someone ought to start one.  I'm
willing to do so if it doesn't already exist.
Regards,
Keith
 

-Original Message-
From: Matthew Bramble [mailto:[EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 10:31 AM
To: [EMAIL PROTECTED]
Subject: [Declude.JunkMail] OBFUSCATION filter - attachment
This is the obfuscation filter attached.  Please don't reply to the
other message or don't include the filter.
Matt
   



---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Matthew Bramble
I've posted a newer version of the OBFUSCATION filter on my site.  This 
contains the removal of the attachment thing and also the removal of 6 
(of over 100) tests in order to be more forgiving, sans the PayPal issue.

http://208.7.179.20/decludefilters/obfuscation/obfuscation_09-14-2003c.txt

If you find any false positives with this besides the Ticketmaster one 
that I've already counterbalanced, please let me know.  I would imagine 
that posting to this group would be better than PM's unless others mind 
having discussion here.  That way everyone would know about any issues ASAP.

Thanks,

Matt

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


RE: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Kami Razvan
Bill:

We have a lot of these well known sites in our whitelist as REVDNS.

WHITELIST   REVDNS  .paypal.com

Paypal has been there for ages, same with eBay, IBM, Oracle, etc.  The
REVDNS is almost foolproof way of letting paypal come through without
worrying about anything.

Regards,
Kami

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Bill Landry
Sent: Sunday, September 14, 2003 3:44 PM
To: [EMAIL PROTECTED]
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


Just an FYI, I've added:

MAILFROM -7 ENDSWITH paypal.com

to the Test Exclusions, as it was flagged by the Obfuscation test.

Bill
- Original Message - 
From: Matthew Bramble [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 12:27 PM
Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 Thanks Bill.  And I've got a few more in me I believe :)
 
 Matt

---
[This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type unsubscribe
Declude.JunkMail.  The archives can be found at
http://www.mail-archive.com.

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.


Re: [Declude.JunkMail] OBFUSCATION filter

2003-09-14 Thread Frederick Samarelli
Do you put these in the Global.cfg
- Original Message - 
From: Kami Razvan [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Sent: Sunday, September 14, 2003 9:04 PM
Subject: RE: [Declude.JunkMail] OBFUSCATION filter


 Bill:

 We have a lot of these well known sites in our whitelist as REVDNS.

 WHITELIST REVDNS .paypal.com

 Paypal has been there for ages, same with eBay, IBM, Oracle, etc.  The
 REVDNS is almost foolproof way of letting paypal come through without
 worrying about anything.

 Regards,
 Kami

 -Original Message-
 From: [EMAIL PROTECTED]
 [mailto:[EMAIL PROTECTED] On Behalf Of Bill Landry
 Sent: Sunday, September 14, 2003 3:44 PM
 To: [EMAIL PROTECTED]
 Subject: Re: [Declude.JunkMail] OBFUSCATION filter


 Just an FYI, I've added:

 MAILFROM -7 ENDSWITH paypal.com

 to the Test Exclusions, as it was flagged by the Obfuscation test.

 Bill
 - Original Message - 
 From: Matthew Bramble [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Sent: Sunday, September 14, 2003 12:27 PM
 Subject: Re: [Declude.JunkMail] OBFUSCATION filter


  Thanks Bill.  And I've got a few more in me I believe :)
 
  Matt

 ---
 [This E-mail was scanned for viruses by Declude Virus
 (http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To unsubscribe,
 just send an E-mail to [EMAIL PROTECTED], and type unsubscribe
 Declude.JunkMail.  The archives can be found at
 http://www.mail-archive.com.

 ---
 [This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]

 ---
 This E-mail came from the Declude.JunkMail mailing list.  To
 unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
 type unsubscribe Declude.JunkMail.  The archives can be found
 at http://www.mail-archive.com.


---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.