[sniffer] Re: Bad Rule: 1604021
Thanks for reporting this, Pete! My numbers were more extreme than Pi-Web's. That bad rule triggered on 18,023 messages yesterday. Due to the rest of my spam software, two-thirds were either passed (as presumed ham) or deleted (as very spammy). So the one-third that was held, I re-scanned today. MessageSniffer today would catch 6,419, and ignore 218. Of the 218 that MessageSniffer would ignore today, 17 are spam and the rest really are ham. Andrew. -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, October 15, 2007 1:00 PM To: Message Sniffer Community Subject: [sniffer] Bad Rule: 1604021 Hello Sniffer Folks, This is an alert about a potentially bad rule 1604021. The rule was an abstract pattern for some of today's image spam. Indications are that the final coding was too broad. The rule was in place for approximately 5 hours ending about 30 minutes ago. Some differences in timing are inevitable since all rulebases are compiled individually. If you have the ability to release and rescan from quarantine based on SNF rule IDs then we recommend executing that process against this rule id: 1604021. Hope this helps, Thanks, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Beta
Pete, I am attempting to get caught on the latest beta and just have a few questions. I noticed Sniffer is now called a different way in the Declude config files, is that correct? On the last release (running persistent), we have numerous entries in the declude.cfg file labeled: SNIFFER-TRAVEL external047 C:\IMail\Declude\Sniffer\WeightGate.exe -12 %WEIGHT% 19 C:\IMail\Declude\Sniffer\snifferlic.exe codehere 20 However, it appears the categories are going away (posted in some previous messages) and there is a since of urgency needed in upgrading as these won't be populated any longer soon. I take it we run the persistent mode the same way, but have a different hook into Declude? Thanks for the aid and understanding. Keith -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, October 15, 2007 10:05 PM To: Message Sniffer Community Subject: [sniffer] Re: Beta Hello Phillip, Monday, October 15, 2007, 6:39:22 PM, you wrote: I just tried installing the beta and it appeared to work too well as it was sending all the mail to the spam box. I am not sure if this was due to the bad rule that was just replaced or something I was doing wrong. The bad rule turned out not to be as bad as we suspected -- so I don't think that was the problem. Also the bad rule has been gone for a while so you're not likely to have it in your rulebase. I am currently running in the persistent mode. I set the xml file to point to the correct paths in the 4 places and started up the sniffer server. I then changed the bat file that the agent calls to run snifclient.exe file licenseID %1 is this the correct format? I am still using an old vopmail 5 mail server. The correct way to call SNFClient.exe is either just like you call the older SNF (compatibility mode): SNFClient.exe authenticationxx scanthis.msg I'm guessing from your notes and memory SNFClient.exe authenticationxx %1 --- The other way you can call SNFClient.exe to perform a scan is simply: SNFClient.exe scantthis.msg --- If you run SNFClient.exe without any parameters it will remind you how it can be used. If you called it with: SNFClient.exe file licenseid %1 Then that would be too many parameters so you would probably get an error and possibly a nonzero result. I may simply be misinterpreting your notes - but if you did something like this in your script (batch file) then it's possible the result would be to hold every message. At the moment I switched back to the old version of sniffer after going through 600 emails by hand and sorting out spam and real mail and manually placing them in the correct mailboxes, that was fun. Sorry about that. I tried to be as explicit as possible in the readme files and the help system in the program itself (running SNFClient.exe on the command line by itself should list all of the ways the client can be called.) The new SNF doesn't have a persistent and non-persistent mode like the old version. Instead, it is strictly a client/server model. Run the SNFServer.exe program as described in the read me and then leave it running. Then, with your message processing script you can call the SNFClient.exe program in place of the old SNF program without any modifications if that is easier -- -the SNFClient.exe will accept the same parameters as the old SNF program without a problem. This makes it relatively easy to switch back (as you did). If you start out running the SNFServer from the command line then it's display will help you to know when things are working correctly -- you will be able to see when messages start going through and you should quickly get an idea of what looks correct. Once you're confident in that setup then you can run the SNFServer using srvany or firedaemon or your other favorite utility that runs programs as a service. Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Beta
Hello Darrell, Tuesday, October 16, 2007, 9:26:47 PM, you wrote: Pete, Can you cover how the communication for the GBUdb system works? Sure. (And documentation is coming along with a web site redesign, so there will be plenty of additional detail on all things new SNF arriving over the next few weeks). Who does it exchange information with and how? Does it need special ports open? I'm going to assume this question is at least in part about security issues so I will frame my response from that perspective: Every minute or so (adjusted dynamically by the system) each new SNF node contacts one of our SYNC servers. The connection is made to port 25 by default. Since most MTAs will regularly make these kinds of connections no special ports will need to be open. If your MTAs are not allowed make outbound connections to port 25 for some reason then you have the option to make the connection to port 80. Our SYNC server software rejects connections by default. If an SNF node follows the expected connection protocols and authenticates properly and consistently then it will be allowed to communicate with the system. If it fails to do any of these things or looks suspicious in any way then it will be automatically black listed for increasingly extended periods and potentially null routed by our fire-walls. The security mechanisms are fully automatic and constantly monitored. The authentication protocol used to identify properly licensed SNF nodes is described in the file AuthenticationProtocol.swf. This file is included in the beta distributions and is also visible in the page linked below. At present there is no mechanism for connections to be made into SNF nodes -- only from SNF nodes to the SYNC servers. Also, there is no mechanism that allows the SYNC server (or any of our systems) to manipulate the SNF nodes except by the protocols described in the GBUdb documentation and by reporting update availability, tuning data, etc. This link helps explain how these interactions work: http://kb.armresearch.com/index.php?title=Message_Sniffer.TechnicalDetails.GBUdb The SYNC system is separate from the rulebase delivery system. All of the data transmitted and received by the SNF nodes is in plain text or base64 encoded. The format of the data is XML. With the exception of GBUdb traffic, the telemetry transmitted to us is available to you directly in the .status. reports made by the SNF engine. Status reports can be found in the same directory as your SNF log files. You can use these XML based posts to create your own real-time monitoring systems. In addition to the GBUdb functions, the telemetry eliminates the need to send in log files by providing near real-time pattern matching statistics; supports virtual spamtraps and other collaborative learning systems; and provides performance analysis, error detection / correction, and system tuning metrics. One other security note -- the virtual spamtrap system can be turned off easily if you wish. Normally the virtual spamtrap system will send us a random sampling of messages that come from the worst known IPs when those messages do not match known pattern rules. In most systems, these are messages that would normally be discarded. Samples are infrequent by design so they should not account for any appreciable bandwidth. Similarly, the GBUdb protocol is designed to share information sparsely so that no appreciable bandwidth or CPU capacity is required. Please let me know if I missed the mark on your questions. Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Beta
You nailed it Pete, Thanks! Darrell -- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Pete McNeil wrote: Hello Darrell, Tuesday, October 16, 2007, 9:26:47 PM, you wrote: Pete, Can you cover how the communication for the GBUdb system works? Sure. (And documentation is coming along with a web site redesign, so there will be plenty of additional detail on all things new SNF arriving over the next few weeks). Who does it exchange information with and how? Does it need special ports open? I'm going to assume this question is at least in part about security issues so I will frame my response from that perspective: Every minute or so (adjusted dynamically by the system) each new SNF node contacts one of our SYNC servers. The connection is made to port 25 by default. Since most MTAs will regularly make these kinds of connections no special ports will need to be open. If your MTAs are not allowed make outbound connections to port 25 for some reason then you have the option to make the connection to port 80. Our SYNC server software rejects connections by default. If an SNF node follows the expected connection protocols and authenticates properly and consistently then it will be allowed to communicate with the system. If it fails to do any of these things or looks suspicious in any way then it will be automatically black listed for increasingly extended periods and potentially null routed by our fire-walls. The security mechanisms are fully automatic and constantly monitored. The authentication protocol used to identify properly licensed SNF nodes is described in the file AuthenticationProtocol.swf. This file is included in the beta distributions and is also visible in the page linked below. At present there is no mechanism for connections to be made into SNF nodes -- only from SNF nodes to the SYNC servers. Also, there is no mechanism that allows the SYNC server (or any of our systems) to manipulate the SNF nodes except by the protocols described in the GBUdb documentation and by reporting update availability, tuning data, etc. This link helps explain how these interactions work: http://kb.armresearch.com/index.php?title=Message_Sniffer.TechnicalDetails.GBUdb The SYNC system is separate from the rulebase delivery system. All of the data transmitted and received by the SNF nodes is in plain text or base64 encoded. The format of the data is XML. With the exception of GBUdb traffic, the telemetry transmitted to us is available to you directly in the .status. reports made by the SNF engine. Status reports can be found in the same directory as your SNF log files. You can use these XML based posts to create your own real-time monitoring systems. In addition to the GBUdb functions, the telemetry eliminates the need to send in log files by providing near real-time pattern matching statistics; supports virtual spamtraps and other collaborative learning systems; and provides performance analysis, error detection / correction, and system tuning metrics. One other security note -- the virtual spamtrap system can be turned off easily if you wish. Normally the virtual spamtrap system will send us a random sampling of messages that come from the worst known IPs when those messages do not match known pattern rules. In most systems, these are messages that would normally be discarded. Samples are infrequent by design so they should not account for any appreciable bandwidth. Similarly, the GBUdb protocol is designed to share information sparsely so that no appreciable bandwidth or CPU capacity is required. Please let me know if I missed the mark on your questions. Hope this helps, _M -- # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Beta
Our SYNC server software rejects connections by default. If an SNF node follows the expected connection protocols and authenticates properly and consistently then it will be allowed to communicate with the system. If it fails to do any of these things or looks suspicious in any way then it will be automatically black listed for increasingly extended periods and potentially null routed by our fire-walls. The security mechanisms are fully automatic and constantly monitored. If something goes wrong on my server, either by a mistake I make in a configuration file or a bug or whatever, and my server in connecting to the SYNC server should be rejected and subsequently black listed, is there a notification that takes place that some one will review to see if that sniffer license is otherwise valid and otherwise no known problems are seen so that I will then be notified saying hey there is a problem contact us so that the problem can be resolved? John T # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
