On Wednesday, June 16, 2004, 12:15:26 PM, Kirk wrote: KM> I've been getting a ton of spam lately that's catagorized as KM> experimental. I've been weighing experimental lighter to avoid false KM> positives. What criteria do you use to move items from experimental to KM> another catagory? Do you want experimentals forwarded to you as well as KM> non-hits?
If we've tagged it as experimental and it is spam then forwarding it to us won't really help since our system will see the message as having been tagged and will filter it out of our incoming spam streams. Experimental rules are those that might be dynamic or that are coded outside of the normal paradigm - for example a broadly coded heuristic that matches a broken header form, or a generalized version of a domain allocation used by spammers. By volume, most Experimental rules are received IPs that are captured from our spamtraps. These rules tend to cause the majority of false positives in this group. While the Experimental group should probably be weighted slightly lower than other groups due to the nature of the content, the group is still very safe and should be weighted high enough in your system to cause a message to be held for review and perhaps deleted after a period. If you don't implement a similar policy mechanism on your system then you will probably have a difficult time working with the experimental rule group - unless you can afford a slightly higher tolerance for false positives due to the nature of ip based filtering. As time goes on, the experimental rule group becomes more reliable due to the policy we use to remove errors. In general, any IP rule that is removed for a false positive report will not be added back into the rulebase without some significant review. As a result, IPs errors associated with larger servers have already been removed in large part. More recent false positives tend to be on a much smaller scale. IPs that are coded into the experimental rule group have at least hit our spamtraps in a verifiable spam message and frequently have also been verified by an alternate source such as SBL, spamcop, etc... I would guess that more than 70% of the IP rules that are coded in the experimental group have at least two reasons to be there at this point. I recommend that you increase the weighting on the experimental group and be aggressive about reporting any false positives that might arise. False positives in the experimental rule groups have been dropping for some time and will continue to do so. If your weighting is based on earlier experiences it is definitely time to revisit those calculations. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
