Hello Richard,
Wednesday, December 31, 2008, 11:49:35 AM, you wrote:
|
> |
Does the snf XML command interface for GBUdb work? I was considering pumping in bad IPs as I find them into the GBUdb and also short-circuiting spam processing by calling the GBUdb to determine the status of an IP to reduce workload. Is this something that sounds like a workable idea? |
It does work:
http://www.armresearch.com/support/articles/software/snfClient/commandLine.jsp
There are some considerations:
* If you mark an IP as bad in GBUdb then it will remain tagged that way until you change it. Instead you might want to give IPs some relatively high bad count so that those IPs can be forgotten over time, or refreshed if you determine they are bad at a later time.
* Blocking connections based on GBUdb has the effect of extending the black-list entry to something on the order of 24 hours if the IP record is based on statistics (ugly, not bad). This is because any good messages that might be received from the IP will not be heard by SNF while the IP is blocked up front. After some period of time the GBUdb statistics will be condensed and the IP may fall back into a zone where messages will be allowed. Condensation happens once per day by default. If a bad message is received again and pushes the IP into the black range then the IP will again be blocked for the remainder of that day (assuming you intend to block connections based on "black" GBUdb results.
* If you are building code to do up-front blocking (as you say short-circuiting processing) then you might want to consider talking to SNFServer directly via XCI instead of using the SNFClient. The SNFClient essentially translates command line parameters into XCI requests. If you make those requests with your code then you can skip the overhead of starting another child process.
* On gateway/proxy systems a very effective method of processing is to use SNF as the first scanner in the chain (during SMTP if possible) and to drive a dynamic local blacklist with it's results to block connections.
In this configuration:
--- SNF can perform full content scans during SMTP without slowing or overloading your system.
--- SNF can provide X- headers for later processing in case you want to add weights for some scores.
--- SNF content scans integrate with GBUdb so that content scanning work is truncated when IPs are bad.
--- SNF results caused by GBUdb overrides are unique and exposed so you only need a single operation.
--- Typically bad content scan results can be used to reject messages and add 30 minute black-list entries.
--- Typically caution results can be used to drive gray-listing for the IP.
--- Typically black results can be used to reject messages and add 1 hour black-list entries.
--- Typically truncate results can be used to reject connections and add 1 hour black-list entries.
In a more advanced system:
* Caution and Black results from SNF scans are very accurate indicators of Spam Storm conditions. If your filtering system can be configured with more than one mode then you can use any Caution or Black entry within the last 5 minutes to indicate a Spam Storm and change your system's condition to the more aggressive configuration. This can be a very effective way to turn on more aggressive gray listing and checking functions for short periods while normally running your system without them to avoid unnecessary support traffic.
* It is possible to integrate the SNF engine directly in your own MTA or Gateway software if you are making modifications at that level or writing your own as some of our larger filtering customers and OEMs have done. This can provide significant reductions in overhead at the expense of isolation. Often other elements in the system represent an overwhelmingly large part of the system load compared to SNF so this extra step may not be needed but on extremely efficient "carrier grade" systems it can represent a useful improvement in performance since child processes and IO operations are eliminated.
Please let us know if you have any other questions.
Thanks!
_M
--
Pete McNeil
Chief Scientist,
Arm Research Labs, LLC.
############################################################# This message is sent to you because you are subscribed to the mailing list <sniffer@sortmonster.com>. To unsubscribe, E-mail to: <sniffer-...@sortmonster.com> To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com> To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com> Send administrative queries to <sniffer-requ...@sortmonster.com>
