Re: [sniffer] My issues with the General category, looking for abettersolution

[Matt]

Greg, The reason why you aren't seeing these is because you aren't weighting Sniffer General at your subject tagging or hold weight, so it takes multiple hits for the false positives to show up on your system.  I assure you that there are in fact many false positive Sniffer hits that only Sniffer is tagging. I'm trying to deal with this in a more proactive way than before on my system by defeating the technical tests such as BADHEADERS, etc. for known bulk mail providers (not spam houses), and establishing a base score according to their trustworthiness.  For instance, ConstantContact, aka roving.com, has a history of servicing spam lists, frequently hitting addresses that are heavily spammed and harvested from whois or from Web sites, but they are also quite popular with small businesses for sending out legitimate newsletters.  One can't safely block all of roving.com, but I establish a base score of 8 and mostly hold on a 13, so something like a SpamCop, SURBL, or Sniffer hit will put them over the top.  I blacklist the offenders and whitelist the false positives.  Sniffer doesn't tag roving.com anymore, but at one time it was being tagged.  Note that AHBL-SOURCES has roving.com tagged as well as some more minor lists. The bigger issues are primarily associated with companies that send advertisements to their customer lists.  The travel companies are one of the best examples of this, and they are frequently blacklisted due to reporting to things like SpamCop, but I have found no evidence of companies like Orbitz, Hotwire, Travelocity or Expedia harvesting addresses or denying unsubscribes.  These companies are also frequently tagged with Sniffer, either in General or Travel.  I assume that the Travel listings are from these companies using third-party services to advertise their services, which is of course spam, but when it comes from the first-party to their own customers, it is not spam by my definition, and as I noted, someone even reported Orbitz as a false positive to me yesterday (Sniffer-General plus my base weight of 8 for the bulk-mail provider, and this has happened before. You might think that my base-weighting of this material is a bad idea, but the problem was that before, I would get random hits on technical tests, and custom filters, and some static spammer RBL's which did little to differentiate between the good and the bad. False positive reports from me to Sniffer have been about 1/4 to 1/3 in the General category in my estimation, but the hit rate for this category on my system is measurably below 10%.  Overall this class of E-mail accounts for over 75% of my false positives (hence the baseline scoring beyond Sniffer as a solution).  False positive identification, research, whitelisting, and reporting to Sniffer is by far my most time consuming process, and I'm trying to figure out how to streamline it.  What I know wasn't working was leaving things just simply weighted the same as everything else and seeing the same senders sometimes be held and sometimes get passed, and reporting the false positives on these sorts of things only offered a temporary resolution in many cases. Matt System Administrator wrote: on 12/15/04 11:41 PM, Matt wrote: I've been having a lot of issues with false positives in the General category, and I'm in search of a better way to handle such things after making little progress without a large time commitment to the issue that this creates. Wow, I'm not seeing anything close to what you are reporting. We had 976 messages fail sniffer-general yesterday. We don't hold messages but mark them as "spam" in the subject message with weight 30 - 39 and delete them at 40 or more. I looked at all the messages that failed sniffer-general and had a weight from 30 - 80. Of those 72 messages I only see one message that probably wasn't spam. It had a weight of 31 so it only had spam added to the subject and was delivered to the recipient. All the messages with a weight of 40 or more that failed sniffer-general were indeed spam and were deleted immediately. Greg This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- ===================================================== MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =====================================================