Thanks Pete - I'll save that command. I also suggest that some of your instructions might be helpful to see in the documentation in the chapters on how to deal with false positives.
From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Tuesday, October 07, 2008 3:41 PM To: Message Sniffer Community Subject: [sniffer] Re: GBUdb False Positives vs. Rule IDs Hello Andy, Tuesday, October 7, 2008, 2:40:01 PM, you wrote: > Hi Pete, >> You can drop the record for the IP from GBUdb with SNFClient -drop <IP>, but if the system is not configured properly then the IP will quickly rise back into the truncate list. << The IP address in question was a third party IP address, not related to us, not a gateway. It was not in the ignore list and shouldn't be - does that qualify as "configured properly"? Yes. > >> If that is being caused by a pattern rule then you need to discover the pattern rule from logs first and then panic that rule and report the FP.<< Hm - so if we have such a GBUdb FP issue, we would need to first go into the log for the message ID in question and locate the IP address. THEN, we have to search the log files to find where this IP address may have occurred (possibly several days of logs, before someone noticed legitimate email from being missing) in hopes of eventually still finding some log entry that relates to the original rule-ID, before we can add it to the panic list? Yes-- more or less. It's not as bad as it seems though. In order for an IP to get into the truncate range in GBUdb it has to consistently send messages that match pattern rules. That is 95% of the time if a message is sent from this IP it matches a pattern rule AND it has to send a bunch of them. If the messages come in over separate days the statistics condense every day -- so on any given day it is likely a number of messages would have to come in and match pattern rules. That means that a message matching the offending pattern rule is likely to be listed in the same log file and previous days (if any). It also means that if you find that IP in that log you are virtually guaranteed that the message you find will have either matched the pattern rule or been truncated. In this case the probability figure is 1 indicating that all messages from this IP have matched pattern rules. GBUdb override results (caution, black, truncate) do not change IP statistics... so the only way for an IP to get into the truncate range is by consistently producing messages that match pattern rules. Presumably if substantially all messages from this legitimate source were to be tagged as spam then they would be reported as false positives. Even if they were not immediately reported as false positives then the daily condensation of GBUdb statistics would force the IP out of the truncate range until more messages were tagged by the pattern rule -- and presumably one or more of those would be reported as false positives. Bottom line -- it should not be difficult to find log records associated with this IP that are also associated with the pattern rules that pushed it into the truncate range. > I suppose it would be technically impossible to include the underlying rule in the GBUdb, so that it can be properly reported when messages are blocked? Yes. The GBUdb engine only stores the statistics about the IPs and the data needed to index and access these records quickly. However, as I've said, information on the pattern rules should be relatively easy to find -- especially for truncate cases. > >> Are you reporting such an FP?<< Yes, your FP support identified the underlying rule and reported it back to me. Of course, I need to have a panic procedure in place that doesn't rely on outside assistance. Doesn't happen often, but better ask the questions now than when the brown matter hits to air circulation enhancer. This case is somewhat unique. The pattern rule has been around for a very long time -- so it is extremely unlikely that a similar case would arise again. A short-term and immediate fix for such a case -- while figuring out what is really going on -- is to reset the statistics on the IP so that they are not in the truncate range and so that it would take a large effort to get them there. For example, you could SNFClient -set <IP> ugly 0 32 This would move the IPs statistics far toward the white so that a truly large number of hits would be required to push it back into truncate even if every message failed a pattern rule. In the mean time the IP would be in the "normal" range. This gives you immediate relieve with a "fire and forget" command. The GBUdb statistics for the IP will eventually return to the correct value for the IP and by the time that happens you will have resolved the underlying pattern rule issue or made some other decision regarding the IP. Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. ############################################################# This message is sent to you because you are subscribed to the mailing list <sniffer@sortmonster.com>. To unsubscribe, E-mail to: <[EMAIL PROTECTED]> To switch to the DIGEST mode, E-mail to <[EMAIL PROTECTED]> To switch to the INDEX mode, E-mail to <[EMAIL PROTECTED]> Send administrative queries to <[EMAIL PROTECTED]>
