Here's some stats, very typical, for Monday morning, 00:00 through about
08:00 :
1 ACL mta_clients_relay
2 SMTP unauthorized pipelining
3 RBL orbs.dorkslayers.com
5 ACL mta_clients_pipel (pipelining)
5 RBL dialups.relays.osirusoft.com
10 Other
11 RBL rbl-plus.mail-abuse.org
13 RBL korea.services.net
14 ACL unauthorized relay
15 RBL dynablock.wirehub.net
18 SMTP invalid [EMAIL PROTECTED]
61 RBL proxies.relays.monkeys.com
77 ACL from_senders_nxdomain
83 DNS timeout for MTA PTR hostname (forged @sender.domain)
92 RBL blackholes.wirehub.net
128 ACL body checks
143 SMTP Bad HELO
168 RBL bl.spamcop.net
194 RBL relays.ordb.org
232 ACL mta_clients_slet
292 ACL mta_clients_dict
302 ACL mta_clients_hel
365 DNS nxdomain for MTA PTR hostname (forged @sender.domain)
457 ACL header checks <<<<<<<<<<<<<<
547 ACL from_senders_black
1199 DNS no A/MX for @sender.domain
1751 ACL to_recipients_dead
2857 ACL from_senders_slet
2912 ACL mta_clients_rbl
2956 ACL mta_clients_bogus
3038 ACL from_senders_clueless
3314 ACL mta_clients_blaksender
7558 ACL mta_clients_dead
=========================================
28824 TOTAL
The header_checks filter has about 300 rules in it.
My point is not be fooled into thinking that you are making a significant
contribution to reducing spam by spending time on header filtering rules.
Above hit rate of 500/29000 = 0000000.017% :))
In my experience, it's mostly a waste of your time AND, due to the low
percentage of total rejects, it's a waste of your Imail CPU time, since
scanning headers is quite CPU intensive. ok, ok, you spent $5k for your
quad Xeon box for 5000 msgs/day and you might as well do something with a
machine that is 99.99% idle. :))
Here's a technique that will help you gain a perspective on the situation:
1. Use Imail's "copy all mail" feature to send all mail to a program alias,
which is just simple script with grep.
2. For grep for Win32, get it here: http://unxutils.sourceforge.net/
3. Somebody here will have to show you how to put a program alias in Imail.
4. This script works on *nix for stripping the Subject: headers and just
storing them, rather than the entire message bodies. It, or something very
similar, should work on Imail:
the script harvest_bcc_headers.sh, or harvest_bcc_headers.bat, contains:
/path/to/grep.exe "^Subject:" >> /tmp/harvest_bcc_headers.txt
The extracts only lines the start with "Subject:" to the .txt file.
btw, the Win32 grep syntax is different from the above unix grep syntax.
5. then run this pipeline of commands, using the other tools in the unix
tools package, wraps :
grep "^Subject:" < /tmp/harvest_bcc_headers.txt | sort -f | uniq -ic | sort
-rf | less
6. .... to obtain a quick report like this, left column showing total hits
per harvested headers file:
544 Subject: Open House Report for Sunday August 25, 2002
361 Subject: Transport
239 Subject: If you don't come see us, you're missing something!
228 Subject:
217 Subject: Returned mail: User unknown
167 Subject: Receipt
150 Subject: #22896 Unclaimed Funds Notification
148 Subject: (no subject)
127 Subject: Mother finds 71K in 15 Year Olds Closet
125 Subject: Returned mail: see transcript for details
112 Subject: RE:
106 Subject: Doppler 10 Pinpoint Weather
96 Subject: FW: (no subject)
94 Subject: failure notice
81 Subject: Re: also
80 Subject: RE: Running out of time.
70 Subject: Lowering the threshold to $25 for Free Super Saver Shipping
70 Subject: Listing Confirmation--Amazon Marketplace
70 Subject: FW:
68 Subject: Ticket Confirmation - EveryDayPlay
66 Subject: RE: Hi
65 Subject: Re: Draft Eats
64 Subject: Making History: Live Game Webcast Monday
61 Subject: Delivery Status Notification (Failure)
58 Subject: HI
55 Subject: Returned mail: Service unavailable
52 Subject: RE: [ohio-state] Cool
50 Subject: eSquire results
49 Subject: Autoreply: eSquire results
48 Subject: Today's Headlines from NYTimes.com Sunday, August 25, 2002
48 Subject: Pages That Will Make You Feel Good!
48 Subject: FREE Thermal Coffeemaker!!
47 Subject: Webshots - Wildlife
46 Subject: FlightFund News
44 Subject: I know something good about YOU!
43 Subject: Exit42: SLACKER Message
42 Subject: Undelivered Mail Returned to Sender
42 Subject: Friendship is a Blessing = )
41 Subject: Notification of an Instant Purchase Payment Received
41 Subject: Fwd: (No subject)
41 Subject: A Sunday Morning Treat Especially For You
40 Subject: Who Needs A Man?
40 Subject: Pill Potential - RealAge Tip of the Day
40 Subject: Get the Best Deal EVER on your New or Used Car!
40 Subject: **21st century teacher****8/25/02
40 Subject: Does anyone owe you MONEY? You can check in SECONDS
40 Subject: Closeout Bargains - Save Over 75% Today Only!
39 Subject: Re: [frenchbulldogz] Re: Pointer color
38 Subject: OSU 45, Texas Tech 21 -- Xtras and O's newsletter
38 Subject: INTRODUCTIONS + FLOWGO's MOST DISGUSTING AWARDS
38 Subject: Hoooooooooowee! E-HOWDY! :o)
36 Subject: Weekend Surf'n Go Sports
The above report was a file /tmp/harvest_bcc_headers.txt with nearly 40K
headers in it.
Gotcha: A weakness in storing only the headers is that when you see a
suspect header with A) "sufficient" number of hits worth inspecting, you
can't B) go read the message body to see if it's really spam, since you
only stored the Subject header.
Gotcha 2: Double-delivering ALL messages to the "copy all" routine is
itself expensive for Imail. So I don't recommend doing it forever, but
only to give yourself a perspective on how unproductive, for you + Imail,
filtering headers is for your machine. esp something really stupid like
carrying a rules file of many 100's or 1000's of rules.
The above report should allow you to respond to a customer's complaint
about a objectionable spam message. Objectionable as it may be, if you
have only received 3 messages today with that header, out of 5,000 messages
today, is it really worthwhile to burden your Imail CPU to scan 4,997 for
nothing, day after day after day?
hth
< of course, scanning messages bodies is MUCH more expensive due to the
MUCH larger body size vs Subject: header size. >
To be comletely fair, the above categories are also filtering on "header"
lines, but on the envelope headers of MAIL FROM: and RCPT TO: :
547 ACL from_senders_black
1751 ACL to_recipients_dead
2857 ACL from_senders_slet
3038 ACL from_senders_clueless
but the difference with the Subject scanning is
1) that the envelope scanning is not a regular expression string matching,
but an indexed file lookup
2) that it is during the SMTP dialog (that is, before accepting the body).
Len
__________________________________________________________________
www.menandmice.com/DNS-training : DNS Training
BIND8NT.MEIway.com : ISC BIND for NT4 & W2K
IMGate.MEIway.com : Build free, hi-perf, anti-abuse mail gateways
To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html
List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/
Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
