Re: Re[2]: [sniffer] Rash of false positives

[Darin Cox]

Hi Pete,   There was a consistent stream of false positives over the mentioned time period, not just a blast at a particular time.  They suddenly started at 5pm (shortly after a 4:30pm rulesbase update), and were fairly evenly spread from 5pm - 11pm and 6am - 10am today (not many legitimate emails came in between 11pm and 6am)...spanning 4 other rulebase updates at 8:40pm, 12am, 3am, and 6:20am.  There were a number of different rules involved, and over 45 false positives in that time period.   Since the problem was evidently corrected with the 10am rulebase, you will probably need to look back at what happened starting with the 4:30pm rulebase.  I doubt looking at the current rulebase will help since the problem has now been corrected, but I'm sure you archive them and can look back to see what process breakdown allowed this to happen.   I'm familiar with the panic procedure, but since there was such a broad base of false positives across a number of rules, adding panic rules for all of them just didn't make sense.  Disabling Sniffer entirely would have been the action we would have taken.   Let me know what you find out.   I completely understand the learning curve with new staff, but the quality of the rules is imperative.  Anything you can do to keep that quality high is much appreciated.   Thanks, Darin.     ----- Original Message ----- From: Pete McNeil To: Darin Cox Sent: Tuesday, November 08, 2005 2:49 PM Subject: Re[2]: [sniffer] Rash of false positives On Tuesday, November 8, 2005, 11:02:09 AM, Darin wrote: > Hi Pete,   The rash of false positives seems to have stopped with the last sniffer rulebase update at 10am ET.  It had started with a rulebase update at 4:30pm ET yesterday, and continued through the updates at 8:40pm, 12am, 3am, and 6:20am today.   I'd still like to know what happened, and how we can avoid it in the future.   I've been bound up in some performance tuning today so I've not had a chance to follow this thread until now. When I first looked in on it I scanned the false positive submissions and almost none of them matched any active rules. I know that a couple of rules were pulled out after review last night late .. they had been picked up by some FPs in SURBL & others that matched up with spamtrap submissions. It's possible that these are what you experienced. I won't know unless you can give me some log entries to go with those messages since those entries will tell me the rule IDs. As for having it happen again - that's very unlikely since ever time we pull a rule out due to FPs or potential FPs (the rules that were pulled had not caused any FPs yet but were expected to... one was rr.com IIRC, it was pulled only a couple hours after it's creation). A lot of things have to go wrong to cause an FP problem like you are reporting. Please look up our rule-panic procedure which is designed to mitigate these problems immediately for you if they happen: http://www.sortmonster.com/MessageSniffer/Help/FalsePositivesHelp.html#RulePanic We can't guarantee that rule-panics won't happen, but we can make them exceedingly rare and non-repeatable. I will be processing your FP submissions shortly. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html