Re: [Declude.JunkMail] blocking spam faked as coming from local address

[Matthew Bramble]

Let's keep in mind that the discussion has changed from the original topic of MAILFROM Forged to VERP + Forged. For the last day I've been filtering using the SPAMDOMAINS method which captures examples of both topics in this thread, however it didn't capture E-mail that fakes a local domain when it is sent from my Microsoft SMTP server because I have that IPBYPASSed (there would otherwise be a lot of this). MAILFROM Forged ----------------------------------- As far as the MAILFROM test goes for finding faked local addresses, here are my results but bear in mind that this excludes intra-server faked domains from Web sites:     3 - Spam w/Forged address (2 passed filters with 80% of fail weight, 1 failed).     9 - Legit w/Forged address (E-mails sent from one local user to another local user but didn't use my server for sending.) =========================================================================   12 - E-mails caught with whitelisting local Web server. For me the FP rate of a MAILFROM ENDSWITH local domain test was 75% with whitelisting (as it is currently set) or about 89% without whitelisting because of mail sent from local Web sites.  The FP rate would definitely higher on weekdays because legit volume is higher and several customers have business communications sent forged.  This test tagged a total of 3 pieces of spam out of a total of 1,968 unique messages received (0.15% of unique messages). I am going to look at an entire week's traffic with the MAILFROM test as Andrew suggested in order to spot the possibility of adding a point or two if there is leeway in the current scoring.  For such a small number of forged addresses though, I don't want to risk the possibility of FPing on anything.  I do have problems with legit E-mail doing this that fails multiple tests that I don't want to turn down to allow this, and I don't like to whitelist if at all possible. SPAMDOMAINS-based VERP + Forged ------------------------------------------------ Now as far as the SPAMDOMAINS-based test results go, here's what I found: 120 - Spam messages caught (71%)           117 - Spam w/VERP               3 - Spam w/Forged addresses   50 - Legit messages caught (29%)           41 - Legit w/VERP             9 - Legit w/Forged addresses ==================== 170 - Total Messages Caught The only spams that got through were the two mentioned above that actually forged the local sender.  I also had one false positive in this group which was sent from Yahoo Groups and FP'd because for some reason, this message failed EASYNET-PROXIES.  I assume that this was a problem in the lookup returned by Easynet because that IP is not currently in their database, and that same server successfully sent about 40 other messages without being caught.  This message was also sent to a dead address that I am scoring as a 'spamtrap' but it is forwarded to another account so I'm not killing the message automatically. >From looking at the spam using VERP, almost all of it came from a small handful of companies who have been tagged by FIVETEN-SPAMSUPPORT, MAILPOLICE-BULK, SPAMCOP, EASYNET-DNSBL and SBL.  All but about 5 of these were tagged by at least two of those mentioned which is enough to fail any message with no other points necessary.  None of the spam VERP messages passed my filters. It appears that all of this VERP stuff comes from gray-spam (for lack of a better word).  These are addresses harvested primarily from contest and free membership sites with participants knowingly giving their addresses away for such things (not all of it uses VERP of course).  The ones using VERP likely have somewhat static addresses and therefore these mailers are easily tagged by the leading blocklists.  I don't believe I have any problems with VERP spammers, though this will take more monitoring to make a solid conclusion. I do have problems already with FP's on legit opt-in advertising, some of which use VERP.  Too often such places find their way onto MailPolice or SpamCop only to be removed shortly thereafter, a problem that originates from spamtraps that were once real accounts being forwarded and from some members of these sites that consider anything that is ad-related to be spam, even if they are a customer and have the ability to easily opt-out.  This very fact accounts for the vast majority of my FP's, though they are the types of FP's that I do nothing with because they aren't missed, but I don't want to block them with any sort of regularity. Right now I don't see any opportunity in scoring VERP, and only a very small opportunity in scoring true forged From addresses.  There is no doubt that your test finds spam, but the overall FP rates of both tests scares me greatly.  Everyone's setup is unique though, and so is their traffic,  so some might benefit from the test you are using.  Is that a fair enough presentation?  BTW, are you using grep and other utilities on Windows?  If so, where did you get your tools?  This could make pattern matching much less laborious for me, but I'd have to brush up (a lot) on regular expressions. Matt Bill Landry wrote: Scott, is this list moderated?  I sent a response to the list regarding this thread on Friday and it has not shown up on the list.  This has happened to me at least three times over the past month or so.   Matt, the addresses you are referring to below are not bounce messages, they are Variable Envelope Return Path (VERP) addresses that some list servers use to manage bounce messages and automate address removal.  And just because lots of spammers are starting to implement and support VERP on their spam lists does not mean that I want to deliver these spam messages to my customers.   Do the subjects shown in the attached text file (zipped to pass spam filters) that have been flagged so far today (by the FORGED-DOMAINS spamdomains test I setup) look like legit, non-spam messages?  Over 90% of these messages are sent by VERP style from addresses, but so what, they are still clearly spam that my customers do not want.   Anyway, what works for me in my battle to fight spam may not work for you, and vise versa.  BTW, the search string I used to output this file is shown at the top of the attached file.   Bill ----- Original Message ----- From: Matthew Bramble To: [EMAIL PROTECTED] Sent: Friday, September 19, 2003 11:05 PM Subject: Re: [Declude.JunkMail] blocking spam faked as coming from local address Ok, I set up a test using SPAMDOMAINS functionality as was described in this thread.  It just caught two E-mails, however both were not actually forged, but instead the HELO From address included a long string for list washing of bounced addresses.  One of these is in Bill's list in fact: Received: from mail03-art-edu.mx07.com [209.66.76.42] by my-hosted-domain.com with ESMTP   (SMTPD32-7.13) id A10141101A8; Sat, 20 Sep 2003 01:09:21 -0400 Received: (from [EMAIL PROTECTED]) by mail03-art-edu.mx07.com (8.8.8/8.8.8) id AAA89631; Sat, 20 Sep 2003 00:18:39 -0400 (EDT) Date: Sat, 20 Sep 2003 01:07:37 -0400 (EDT) Message-Id: <[EMAIL PROTECTED]> From: The Arts & Education Source <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Subject: [16] What is Phentermine? MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="MIME_BOUNDARY-23113-0-1064028662" X-Declude-Sender: [EMAIL PROTECTED] [209.66.76.42] X-Declude-Spoolname: De101041101a8865d.SMD X-Note: This E-mail was scanned by iGaia Incorporated's E-mail service (www.igaia.com) for spam. X-Note: This E-mail was sent from mail03-art-edu.mx07.com ([209.66.76.42]). X-Spam-Tests-Failed: SPAMCOP, MAILPOLICE-BULK, NOLEGITCONTENT, FORGEDASLOCAL, W-HIGH, W-MED, W-LOW, W-SUB [16] When I assembled my stats I worked from the E-mail's From address found in the headers and not in the HELO (X-Declude-Sender).  It appears that setting this up using SPAMDOMAINS will result in scoring any bounce handlers that include the receiver's address in the HELO data, but not necessarily in the message's headers.  Our tests were not checking the same things, and it appears that much of what you are catching are bounce addresses.  I was using MAILFROM with ENDSWITH which doesn't catch these bounce addresses.  I double checked, and forged senders continue to be very rare on my server.  I don't know that I want to punish bulk-mailers that are looking for bounces either since many in fact are legitimate, such as several from a recent post referenced by Andrew: im31877-errors+863709.954008572+[[EMAIL PROTECTED] OWNER-NOLIST-20030821115189*[[EMAIL PROTECTED] spencer1-errors+860989.389749042+[[EMAIL PROTECTED] realestateweekly-text-return-44-[[EMAIL PROTECTED] IMCEAEX-_O=CARLSON_OU=NATRVCN018-TORONTO_CN=EXRECIPIENTS_CN=[[EMAIL PROTECTED] sentto-4331469-1096-1061380934-[[EMAIL PROTECTED] Maybe I set up my test wrong (just one domain.tld per line)?  If not, it's probably important to know that you are adding scores to these things.  SPAMDOMAINS works as a CONTAINS filter and not an ENDSWITH filter, so it's going to get tagged all the time with bounce messages instead of forged local senders. BTW, I found the forged E-mails by searching for "@my-local-domain.tld [" since that is unique formating for the X-Declude-Sender line. Matt