[sniffer] Re: Our IP got listed on GBUdb Truncate
On 11/2/18 11:52, Daniel Bayerdorffer wrote: > > Is there anyway for us to see what the offending email was that got us > on the list? Or some other data point to help us clean up our system? SNF doesn't leak message info -- With the exception of auto-sampling of spam (truncated messages, and only if you have it enabled) we don't see message content. What we do get are anonymous statistics and training data. The good news is that you are running SNF, so you can scan your messages and identify any content that might have triggered SNF. Truncate is trained by counting good and bad events -- bad events are when a message matches spam/malware patterns. ... so you can actually check with your own scanner. Truncate is completely automated... so we can't change the list data. It actually doesn't come from a database but rather by skimming the telemetry for these events. In effect the reputation for any given IP resides in each SNF instance around the globe and the truncate list works by eves-dropping on the conversations between those nodes as they "discuss" IP reputations. If the IP is still listed and you send a note to support with the IP requesting a trace then we can collect some events with timestamps. That may help you track things down -- but since you're an SNF user you would probably do better with your own scanner. Hope this helps. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Happy Holidays!
This is just a quick note to let you all know that we're thinking of you. On behalf of the whole team: We wish you a Merry Christmas and a happy, prosperous New Year. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Rulebase refactoring
Hi Message Sniffer folks, Over the past few days we've refactored the databases we use to manage our rulebase. As of about 1600e you should notice that all of the rule IDs in your system are significantly smaller and completely different. Unfortunately, during the transition there were several unforeseen problems that introduced delays and other disruptions. We apologize for the inconvenience. All is well now. Thanks, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Reminder - the Rule Panic feature
Hello Sniffer Folks, In light of today's bad rule event I've discovered that many of you are not aware of the rule-panic feature. The rule panic feature has been built in to the Message Sniffer engine for many years now, and I suppose is used so rarely that folks have forgotten about it. The feature allows you to render any single rule inert immediately without disrupting anything else in the system. So, it could have been used to mitigate this event without taking more drastic measures. Here is a link to the QA article about the rule panic feature: http://www.armresearch.com/Documentation/QA/ltrulepanicsgt-628138610.jsp Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Bad Rule Alert 2654821
Hello Message Sniffer folks, This morning a dormant rule from 2009 was reactivated when new messages reached our spamtraps this morning matching the rule. Unfortunately rule 2654821 causes a high rate of false positives in our current year that it apparently did not cause back in 2009. Since the rule was not recently coded and had been in the system for so many years our monitoring systems did not immediately detect the rule as a false positive case. However, the team did discover the problem after a few hours and removed the rule. This is the only time an old, reactivated rule has caused significant false positive cases -- so it is an exceedingly rare event. None the less we are in the process of reviewing our tools and processes to improve our sensitivity should any similar event occur in the future. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: rule panic not working
On 12/29/2016 08:55 AM, Daniel Ivey wrote: Thanks, but it appears that my server is failing multiple 54- rules. For example from Google, it is failing 54-8064853-304-318-m and 54-8064853-0-2423-f while from Yahoo it is failing 54-8064853-2063-2077-m and 54-8064853-0-3703-f. That is in fact a single rule hitting in multiple places. http://www.armresearch.com/Documentation/QA/ltmatchesgt-1193870513.jsp The rule ID is 8064853. The rule has been removed. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Error Code 69
On 12/15/2016 07:04 AM, Don Winsauer wrote: I have had 419 occurrences of this error since the 1st of the month. I don't even run a virus scanner on this Windows mail server. We are running IMail, Declude with Sniffer. This could be an indication of a file system problem? The only reason that occurs is when the file system / OS prevents SNF from removing the original file. Are the files still there? What changed since the 1st? (did the problem begin then precisely?) _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Error Code 69
On 12/14/2016 06:12 PM, John Tolmachoff wrote: When SNF is configured to inject headers it does so safely--- First, it reads the entire original message into a buffer, then scans it,... Then it writes a new copy of the message to a .tmp file with the headers injected. When that completes without errors, it deletes the original and renames the .tmp file in it's place. That way, if anything goes wrong, the original is always there as a backup until the last moment. The error above indicates that SNF was trying to delete the original message file so that it could move the new version over. Something went wrong and it wasn't able to do that. On windows systems this is most likely because something removed the file first -- perhaps a virus scanner or some other process. On linux systems it's usually a permissions issue (but this does sometimes happen on windows in rare cases). So, if you can figure out what is preventing SNF from deleting the original file you will solve the problem. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: DEB Packages
On 12/01/2016 02:07 PM, Daniel Bayerdorffer wrote: I see that the DEB packages for Message Sniffer are for Ubuntu 14.04. Will these work with 16.04? They should -- there haven't been any significant changes in SNF nor in the parts of Ubuntu that SNF cares about. Still, the packages are considered experimental (mostly due to a lack of exhaustive testing) so be ready to roll back just in case; and do share your results with us. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: .smd.tmp files being left in proc\work folders
On 08/09/2016 05:11 PM, Don Winsauer wrote: These file are being left and not being delivered. They are usually over 20mg. Something is preventing SNF from renaming the file. Find out what it is and then prevent it from blocking SNF. Perhaps a virus scanner has the file open when SNF comes by to rename it?? SNF tries to do everything safely, so when injecting it's headers it writes the new message file with a .tmp extension and makes sure that succeeded before it removes the original and then renames the .tmp. My guess is that the new .tmp file catches the attention of some scanner or other program and that when SNF goes to rename the .tmp file to replace the original it is unable to do it. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] SNF Engine Update to 3.2.1 / Short Buffer Bug Fix
Hi Sniffer folks,
Today we have released a new SNF engine with a minor bug fix. Please
update your SNF installation at your convenience. Chances are that
you've not seen any problems from this bug. If you have experienced
problems they most likely presented as very rare, random errors
possibly causing a crash.
As with most SNF engine updates the simplest process is to replace
your binary with the latest. For windows users here are some links
to the latest engine:
http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox32-3.2.1.exe
http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox64-3.2.1.exe
Simply stop your SNFServer, swap in the new .exe (renamed of course)
and restart SNFServer.
For folks running linux platforms the packages and source tarballs
on our web site have all been updated.
OEMs using the windows SDK should upgrade to the latest DLL which
should be a swap-in replacement.
http://www.armresearch.com/Downloads/index.jsp
---
Technical details:
The bug fix is for a short buffer allocation in the
codedweller/configuration.cpp module. The bug fix also solves
problems unrelated to SNF where applications using the
CodeDweller/configuration engine to parse unusually large XML
attributes could cause a stack overflow.
The solution allocates the buffer for attributes from the heap
instead of the stack and eliminates a short-by-one allocation error.
Those curious about the source code can see the important diff here:
Best,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
[sniffer] Re: [Alligate]Alligate and Sniffer again (NL)
On 01/18/2016 07:26 AM, Bonno Bloksma
wrote:
Hi,
Ok, downloaded Alligate trial, installed in on
a 2012 R2 server.
Made a local dns “server” (resolver) on the
machine but I am not sure if I need it now that we can use
the Google dns server by default.
A local resolver will speed up your bl lookups quite a bit since
they don't have to traverse the network.
How do I hook up Sniffer? I used to have
Declude (and IMail) and had Sniffer connected that way, I
now need to connect sniffer into Alligate.
I cannot find anything in the Alligate Docs I
downloaded.
As far as I know SNF4Alligate still works.
https://www.armresearch.com/Documentation/Papers/InstallGuides/SNF4Alligate.jsp
Best,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: New Version -- SNFMulti 3.2.0 -- Strangers
On 2016-01-04 11:44, Daniel Bayerdorffer wrote: > Are there any other gotcha's I should be aware of? I took a quick look through the tarball and was reminded -- all of the configuration elements are provided as samples after make-install. The instructions say to copy the samples to their correct names and then modify them appropriately-- so that part of it is a manual process. In that case it should be safe to do make install and just skip those steps since your configuration is already happy. All that said; again -- you're really only interested in updating your SNFServer binary. The rest isn't changed. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: New Version -- SNFMulti 3.2.0 -- Strangers
On 2016-01-04 11:44, Daniel Bayerdorffer wrote: > Hi Pete, > > I have a couple of questions about upgrading. We will be upgrading SNF4SA > running on Ubuntu 14.04 with Zimbra email server. > > I previously compiled the source code to install SNF4SA. Can I compile the > latest version and run the make-install to overwrite the existing version? If > so, do I need to re-apply our license information to the configuration files, > etc.? I think that should work. The only thing that's really different is SNFServer. You could stop after make and then replace your SNFServer binary leaving everything else in place. I don't think there are any other gotchas. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: [BULKMAILER] [sniffer] Windows SDK with SNFMulti 3.2.0 -- coming soon.
On 2015-12-28 11:28, Michael Murdoch wrote: > I AM. The latest Windows SDK is posted. It's exactly like the previous one except we changed the version number and the DLLs have been updated. They should be a drop-in replacement for the previous DLLs. http://www.armresearch.com/message-sniffer/download/updates/SNFMultiSDK_Windows_3.3.zip Please let us know if there is more we can do. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Windows SDK with SNFMulti 3.2.0 -- coming soon.
Hi Sniffer Foiks, If you're curious about the Windows SDK (DLLs) ... they should be posted in the next few days, but not yet. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] New Version -- SNFMulti 3.2.0 -- Strangers
Hello Sniffer Folks, A new version of Message Sniffer is available. The most exciting new feature for this version is: Strangers. The "Strangers" algorithm replaces the previous White-Guard algorithm. Strangers prevents high-intensity pre-tested spam from poisoning IP reputations in GBUdb and enhances SNF's sensitivity to these kinds of attacks. Once pattern rules begin to match the pre-tested attack the IP reputations quickly climb into the black enhancing all of SNF's learning systems. Normal, but new, IP sources are held to low-confidence reputations for several hours, but after that are allowed to develop normally. Short summary: Strangers lets SNF close the door more quickly on pre-tested spam while enhancing SNF's learning sensitivity to those events and without interfering with normal IP reputation processing. Here are some links: Packages from the LabRats... http://www.armresearch.com/message-sniffer/download/packages/ SNFMilter tarball... http://www.armresearch.com/message-sniffer/download/updates/snf-milter-1.2.0.tar.gz SNFServer tarball... http://www.armresearch.com/message-sniffer/download/updates/snf-server-3.2.0.tar.gz SNFServer 32bit Windows exe... http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox32-3.2.0.exe Not better, but if you _really_ want it ... SNFServer 64bit Windows exe... http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox64-3.2.0.exe Thanks! and Happy Holidays! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: ShortMatch Resolved - Update your SNF software to remain immune.
On 2015-12-03 21:24, Daniel Bayerdorffer wrote: > Just so I understand correctly, can we use the packages to install over a > current installation that was compiled from source? Probably not -- the deployment might not be exactly the same. If you originally compiled from source then your easiest solution will be to use the tarball and compile from source again. Then you can simply replace the executable you have with the new one you make -- everything is compatible and nothing will need to move. If you use the packages you are essentially starting over. The packages are deployed differently than the source instructions. For example, to do the generic postfix integration with SNF Server you would need to install two packages: the snf-server_ package and then the snf-server-postfix_ integration package. If you wanted to roll your own integration you might just install the snf-server_ package and then build your own scripts and other software on top of that. It's a different paradigm. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] ShortMatch Resolved - Update your SNF software to remain immune.
Hi Sniffer Folks, According to our latest data, the Short-Match FP problem has subsided - most likely due to rule sequestration. We have not seen any significant events in our detection software since 2100e last evening. In the mean time we have updated the SNF software to check for short-match events and treat them as rule-panic events. This renders them inert so that if this kind of rulebase corruption occurs again the SNF engine will be immune. Please update your SNF software to this latest version using the links below. NOTE: The Windows installer is in the process of being redesigned and does not have the latest software. This will take some time. If you are using SNF on Windows and use(d) the installer then use this procedure to update your software: * Stop your SNF service (usually XYNT Service based). * Copy your SNFServer.exe file to SNFServer.old * Download SNFServer-windows-7-prox32-3.1.0.exe (32 bit) or SNFServer-windows-7-prox64-3.1.0.exe (64 bit) and rename it to SNFServer.exe to replace your previous SNFServer.exe. * Start your SNF service. If you were using the 32 bit version (very likely) then replace it with the 32 bit version. There really isn't any difference, but just in case it's simpler to keep things the same. There is no benefit to running the 64 bit version -- It is not faster and is in fact less efficient due to the use of extra large (64 bit) pointers that aren't necessary ;-) Some folks really want a 64 bit version, so we have one. Here are some links to updated versions: http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox32-3.1.0.exe http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox64-3.1.0.exe http://www.armresearch.com/message-sniffer/download/updates/snf-server-3.1.0.tar.gz http://www.armresearch.com/message-sniffer/download/updates/snf-milter-1.1.1.tar.gz http://www.armresearch.com/message-sniffer/download/updates/SNFMultiSDK_Windows_3.2.zip And for the really adventurous: http://www.armresearch.com/message-sniffer/download/packages/ In the packages link you will find all of the latest snapshots and some old ones from our LabRats. The LabRats compile and test SNF for all of the different platforms. You will find RPM and DEB packages as well as tarballs and even the windows stuff that's posted in the updates links above. Be sure to pick the latest version in all cases. It will take a bit of time before all of the ordinary links on our web site are updated with the latest software, so please use the above links instead if you're going to update right now. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Short Match FPs.
Hi folks, Good News! After much research and experimentation we have determined that some time on Nov 28 a corrupted rule entered the rulebase and caused the intermittend short-match problem. We have removed a group of rules surrounding that timeframe and have observed a 3 sigma drop in the rate of short-match events. This indicates that the problem is solved and not likely to return. Now that we know this kind of event is possible (it's not supposed to be mathematically) we will be building a detection and mitigation strategy into the engine... just in case it does happen again. Once in two decades makes that seem unlikely. We will also be continuing our research on the sequestered rules to identify the one(s) that caused the problem and identify a way to prevent that recurring. In the mean time the detection mechanisms we used to monitor our experiments will remain in place so that if we do see any future events we will be able to identify them much more quickly. Sorry for the trouble, Thanks for your patience and continued support! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Short Match FPs.
On 2015-12-01 18:12, Darin Cox wrote: > Thanks for the info, Pete. Appreciate your proactiveness on this. > > Hope you had a good Thanksgiving! Thanks! I did. I'd also like to report that some of our experiments might be showing results. It is possible that the trouble has been mitigated based on the latest data I'm seeing. I will know better how good this data is after about an hour. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Short Match FPs.
Hi Folks, I'm sorry to report there is a problem. For the past few days we have been seeing some intermittent corruption in some rulebase updates. Since we made no changes to precipitate this and since it's only been reported by a few systems intermittently it's a bit of a challenge to nail down. However, it is out top priority at the moment. Here is what we do know about it: The problem appears to have started around Nov 29. It is highly intermittent and random. It causes some false positives. You can identify a short-match event by looking at the index and endex of a rule match. If the difference is less than 5 then you have a short rule match. You can mitigate the problem by temporarily putting the associated rule ID in your rule-panic list in your SNF configuration. Normally the problem goes away on the next rulebase update. Sometimes it doesn't go away but changes the associated rule ID. For now the best thing to do is add a rule-panic entry when you spot one of these. That will solve the problem for that update. Be sure to remove your rule panic entries occasionally since they won't help you after a day. We will continue to work on this until we understand it and it is resolved. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Question, changing from SNF4SA to Milter, using freebsd
On 2015-09-08 04:04, P Pruett wrote: > > Interesting, yes, the spamassassin SNF4SA does seem to be able to use > snf-milter instead of snf-server. That's probably not a good way to go. This will cause each message to be scanned twice. Once by the milter and again by the engine via SNF4SA. If you want to use SNF4SA then you should turn off the milter and use SNFServer instead. > On freebsd 9.3 with Sendmail, I did add the milter and restarted sendmail > and its seems to be playing okay. > > Now I turned it on, I am not sure what the snf milter is doing. That will depend on how you have it configured. The milter interface only provides a few options. Your SNF log should tell you what was found in the scan and the snfmilter configuration will tell you what SNF told the milter to do. > Can you point me to some more documentation with details about what the > milter is doing? > From what I saw in the setup file it can Allow, Accept, Retry, Reject That is defined by the milter interface. Milter.org was shut down permanently just recently. That page says this is where to find documentation on milters: http://www.sendmail.com/sm/open_source/download/ > > I was think it might insert information in the header SNFMilter should inject the usual SNF headers if they are configured (they are by default). > > Would be nice if the milter could be somehow be used to promote IP > addresses into a pf table > for the pf firewall to redirect with? That's an entirely different software project. If you want that kind of functionality then you'd do better to use SNFServer/SNFClient in a postfix filter. The filter script could then be modified to look at the results and respond in any way you can code. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Question, changing from SNF4SA to Milter, using freebsd
On 2015-09-06 13:11, P Pruett wrote: > So what "gotchas" do you know that I need to be aware of if I already > have snf-server > setup and I am going to try snf-milter? The two are not designed to work together. It turns out that SNFMilter has the full SNF engine in it so if you have SNFMilter running you should also be able to use SNFClient and things that act like SNFClient such as SNF4SA. This is not something we test heavily though because almost nobody tries to do this. Most folks who run SNFMilter either build their own software to manage messages (Milter API is highly restrictive) or have SNFMilter inject headers that are later consumed by SpamAssassin and other ubiquitous tools so that they can customize their system easily. If you are using SA after SNFMilter, consider simply adding rules that recognize headers injected by SNFMilter and add appropriate weights for SNF's results. This is a common and successful configuration which allows you to reject some messages during SMTP with SNFMilter and then score the remaining messages using SNF's scan results with SA and other tools that are usually bundled with SA. You shouldn't try to run SNFMilter and SNFServer on the same system at the same time. If you have SNFMilter running, the SNFServer "back-end" should already be provided in that service. (Check that XCI is on, it should be by default). In that case running SNFServer would be redundant. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Bad Rule Alert: 6948148
Rule 6948148 was coded as an abstraction to a fake header and was rapidly removed by QC checks. Most systems are automatically removing this rule. The rule coding has been added to our problematic group so that it cannot be reinvented. Due to our auto-panic feature it is likely this rule will not affect most systems. We apologize for any inconvenience. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: milter and smtp auth
On 2015-02-10 14:53, Thomas Klaube wrote: >> I might also point out that white-listing mechanisms generally lead to >> > abuse. > I tend to agree that white-listing is usually not the best solution > > But please consider this case: one of our users tries to relay mail > through our servers and is originating from a Dial-up IP address with > very bad reputation (maybe within "truncate") but is correctly authenticated. > Would you agree that such mails should not be marked as spam or even > discarded (at least not based on IP address reputation)? > My answer in this case is - it depends. Some systems I know of would consider this too high a risk as you've described it. Others would completely agree that any authenticated system should automatically be white-listed. Unfortunately for the latter group this often costs them a lot in clean-up consulting fees when customers get infected. (we see that a lot lately). Since this is a policy based decision, you could take advantage of the GBUdb drilldown feature and teach your SNF to "trust" the IPs that this customer might use. What would happen then is that SNF would not be able to identify the source IP and so only the pattern matching engine would apply. http://www.armresearch.com/Documentation/QA/ltdrilldowngt--468945561.jsp Effectively you'd be telling SNF not to worry about the IP address for this customer (or for that matter any of the IPs used for dialup by the customer's provider)... only pay attention to pattern matches. That's still making a hole,... but it's your hole and you know why you made it. It's also a pretty small one because if some known spam or malware comes from there it will still get tagged -- maybe not as efficiently -- but it will still get tagged. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: milter and smtp auth
On 2015-02-10 11:23, Thomas Klaube wrote: > Sometimes we see false positives from some of the users although > they have been authenticated correctly. Is there a way to tell > SNFMilter to whitelist authenticated users? There is no such mechanism in Message Sniffer at this time. I might also point out that white-listing mechanisms generally lead to abuse. For example, much of the worst malware these days infects a machine, gain's authentication through email and other systems, and then uses the authenticated accounts to spread itself further -- this vector takes advantage of social hacking (trust of known friends/peers) and hard security hacking (by gaining access to secured accounts the old fashioned way, by stealing the keys). We don't get many requests for this kind of thing -- I'm pretty sure this is the first time I've heard this one. SNFMilter is distributed as source code so you certainly could code this modification yourself if you need it for your system, or you might use a different milter to force acceptance of messages that you've whitelisted either by list or by behavior. Please if you do find a false positive do report it to us so that we can adjust the filters appropriately... much better to get the filtering right than to make holes in it. For reference: http://www.armresearch.com/Support/falsePositives.jsp Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Adding Message Sniffer to Zimbra
On 2015-02-10 01:20, Daniel Bayerdorffer wrote: > But there are no headers in the messages showing snf's results. I can see > that the snf4sa.cf has it set to add them though. > > # Header line containing the results from SNFServer. > add_header all SNF-Result _SNFRESULTTAG_ > add_header all MessageSniffer-Scan-Result _SNFMESSAGESNIFFERSCANRESULT_ > add_header all MessageSniffer-Rules _SNFMESSAGESNIFFERRULES_ > add_header all GBUdb-Analysis _SNFGBUDBANALYSIS_ > > Do you have any more suggestions? Unfortunately, some implementations of SA are hiding these headers. We've seen this a few times recently. There doesn't seem to be a way around it outside of hacking SA itself. (A few people have done that,... but it was ugly). If you want to be able to more easily associate SNF logs with messages you might consider changing SNF's message identifier to use the Message ID. http://www.armresearch.com/Documentation/QA/ltidentifiergt-2021367617.jsp _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Adding Message Sniffer to Zimbra
On 2015-02-09 16:23, Daniel Bayerdorffer wrote: > libpthread package they have listed for 14.04. But the config script still > can't find that library. Can you offer any advice? apt-get install build-essential seems to be the equivalent of CentOS yum groupinstall "Development Tools" which usually solves this problem for redhat variants. Give that a shot and see if it fills in the holes. Usually by the time I've got g++ up and running on ubuntu it "just works" -- hopefully that's not broken in 14. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Adding Message Sniffer to Zimbra
On 2015-02-02 19:53, Daniel Bayerdorffer wrote: > Does anyone have any advice or tips for adding Message Sniffer to > Zimbra 8.6? Specifically with Zimbra's implementation of spam assassin? The SNF4SA plugin included with the Linux source code distribution should do the trick. SNF4SA looks to SpamAssassin like any other SA plugin. It creates a temp file of the message, calls SNFServer to scan the message, and then processes the results in a way SA expects so it can be scored. It _should_ be as easy as that. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Report one off spams
On 2014-12-16 13:59, John Tolmachoff wrote: > When sending occasional one off spam not caught to spam@ would it help to > attach the original headers and source of the body as text files to the > forwarded email? Not usually -- that would complicate things. If we can get the original message in it's original form (like redirecting (not forwarding) from Thunderbird) that would help. However, if simply forwarding from some other client, the less "extra" done, the better it will be. On our end we have to plow through a lot of different formats and sources, so the simpler everything is the more we are able to decipher what we're looking at and locate useful artifacts & structures. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Bad rule report 6237276
Hi Sniffer Folks, A short time ago Rule 6237276 was detected on our conflict instruments and removed from the core rulebase. The rule was in place from approximately 1130 to approximately 1400. We recommend that if you have the ability to release messages matching this rule from your quarantines and rescan them then please do so. The rule was coded to catch a variant of the "/goo.gl/ link" spam and was coded too broadly. The rule was removed when we identified it on our IP/Rule conflict instruments and reexamined it. It's signature on the conflict instrument is already falling dramatically and we expect that most systems auto-panicked the rule making it inert automatically. We are very sorry for any trouble. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Saccades anyone?
On 2014-02-18 17:02, Daniel Bayerdorffer wrote: Any plans to modify the milter code to this in the future? Yes. All platforms will be updated shortly. In fact, if you wish, you can download the snfmulti source from our SVN server and then recompile your milter with the new code. Here is a link: Examine it here with websvn https://svn.microneil.com/websvn/listing.php?repname=SNFMulti Get the source here via svn https://svn.microneil.com/svn/SNFMulti/trunk/ Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Saccades anyone?
Hello Sniffer Folks, We are preparing to release a new version of the Message Sniffer engine that includes an exciting new technology. The "saccades engine" allows SNF to intelligently skip large portions of most messages without missing any important content. The engine borrows from MicroNeil's synthetic intelligence research relating to visual systems processing and essentially gives SNF a behavior similar to what we all do with our eyes: http://en.wikipedia.org/wiki/Saccade The engine learns where matches are most likely to occur and then applies what it is learning in real-time. This allows SNF to rapidly identify messages of a type it has already seen without having to scan the entire contents. This has the potential to improve scanning efficiency by 90% or more. That is, scanning typical messages can happen with 1/10th the work for a 10x improvement in efficiency. Not kidding, we're actually seeing these results on some of our testbed servers! You may have seen me tweet about it: https://twitter.com/codedweller/status/434020178352148480 If you'd like to get in on the fun early and you are using SNFServer.exe then you can find a copy of the new engine at the following link: http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.1.0.zip To swap it in, * Download and unzip the new engine. * Stop your Message Sniffer. * Rename your SNFServer.exe to something like SNFServer.exe.bakup (always a good idea to keep a backup). * Rename the new engine to SNFServer.exe * Restart your Message Sniffer. Please let us know how this works for you. Thanks! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: increase in missed spam
On 2014-02-05 13:56, Herb Guenther wrote: For the last week or 10 days I have seen an increase in missed spam in Sniffer, Declude seems to be picking it up but I require more than a single hit to filter. Anyone else seeing this? This is what we are seeing. The trend has been toward very high volume spikes. To be clear, the graph shows new spam not yet filtered, so the higher numbers mean higher numbers of new campaigns with higher diversity. Hope this helps. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller
[sniffer] Re: large log.xml files
On 2014-01-22 10:33, Daniel Ivey wrote: I was checking out our Imail servers this morning and noticed that under the imail\declude\SNF folder I have a lot of .log.xml files from Sniffer. Is there a way to turn off these files in Sniffer or at least to have it only store about 3 days worth? If you're not using them you can turn them off: http://www.armresearch.com/Documentation/QA/ltxmlgt--999318835.jsp I also noticed that the size of these files has grown from about 60 megs a day to over 500 megs the past couple of days. Does anyone have any ideas as to why the file sizes would increase so much, I haven't seen an increase in messages. We have seen a very large increase in the number of messages... that might explain it. Still, that's an order of magnitude there so you should take a look at the large files and see if something else is happening. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Bulk / Noisy Rule Group
Hi Sniffer Folks, Some of you have been experimenting with our "Bulk / Noisy" rule group which is currently tagged with code 65. This "above band" rule group matches anything that might be bulk mail, list mail, etc... similar to a popular feature of Postini in the past. As an "above band" rule group it does not train GBUdb, and can only be reliably detected by systems that are set up to look for it in SNF's result data. If you are using it -- then you know you are because you've had to tweak your systems to expect it. This note is to let you know that we will be changing this result code to 100 next Friday. The change is to avoid any conflicts with some existing error result codes before we make this feature available more broadly. If you are curious about this feature let us know and we will be happy to answer any questions you have. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Happy New Year!!
Happy New Year!! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: What is your oldest production CPU?
On 2013-12-27 15:45, Matt wrote: Intel 5400 series Xeon here. But don't forget virtualization. I'm not sure what CPU virtualization does to targeting your code. That's a good point The processor should be specified in the VM profile and if I recall correctly it is typically defaulted to the processor of the VM host. I should look closer at this -- but would like some feedback. Thanks, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] What is your oldest production CPU?
Hello Sniffer Folks, We would like to know what your oldest production CPU is. When building new binaries of SNF or it's utilities we would like to select the newest CPU we can without leaving anybody behind. We're also evaluating whether we should split binaries into a "compatible" version base on Intel i686 (or equivalent AMD), and a "current" version based on Intel Core2 (or equivalent AMD). Please respond here. Thanks for your time!! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Whitelist HOW?
On 2013-11-28 19:55, A wrote: I want to set an ignore list in my MessageSniffer installation, so I can receive FBL complains from major ISP. The directives you've set up will adjust GBUdb training, but SNF pattern rules will still tag messages if they match. Normally if you want to globally white-list a particular sender or a particular mailbox on your system you would do that within other systems that you use to process the messages -- so, for example, you might add a rule to your SpamAssassin configuration to white-list all messages going to a particular address; or, if you're using SNF with a postfix filtering script then you would create some scripting to ignore SNF scan results when that mailbox is the recipient; or if you are using one of the many Windows based email platforms then you would enter an appropriate processing rule in the MTA to bypass all filtering rules etc... If you want your SNF rulebase to have a custom white-rule then we can code that for you -- send a note to support@ describing the custom rule you want coded. Be aware, however, that custom white-rules often have unwanted side-effects including that they can be discovered and abused by attackers. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Milter Version
On 2013-10-31 15:01, Daniel
Bayerdorffer wrote:
I’ve been reading the Install notes, but
one thing that is not clear is that the Milter version is up to
date. Is it current and if not will it be in the near future?
We have several folks using SNFMilter with postfix et al and no
problems.
As far as I know it's up to date :-)
SNF in general is built to be stable and highly available, so most
of the changes over time happen in the rulebase and not in the
engine.
Hope this helps,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: snf plugin question
On 2013-09-10 17:02, Peer-to-Peer (Spam-Filter.com) wrote: Is that the right direction? That would open up the black range a bit. Use caution :-), but have fun. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: snf plugin question
On 2013-09-10 11:15, Peer-to-Peer
(Spam-Filter.com) wrote:
Regarding the section in the
SNF Plugin, we’re currently using the defaults (see below).
Could you give us a refresher: Why are there
2 entry’s for each (white / caution / black)?
For example:
They define corner points on a parallelogram that maps the region in
the x,y space:
http://www.armresearch.com/support/articles/software/snfServer/config/node/gbudb/regions/
The points you listed above define the white region with two points.
These points essentially define a line in the graph.
Everything to the left of that line (in the white region) is
considered to be "inside" the region.
Please let us know if there is more we can do.
Best,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] White-Guard
Hi Sniffer Folks, We've been experimenting with a new machine learning behavior. White-Guard is improving early capture rates for new spam and with it overall accuracy and throughput. For example, one thing we've seen since implementing White-Guard is higher truncate numbers across the network-- meaning that more messages are blocked for having bad IP reputations than before we implemented White-Guard. Here is a new blog post that explains what White-Guard is and how it works: http://www.lifeatwarp9.com/2013/08/lies-machine-learning-and-blackhatzes/ You DO NOT need to install or change anything to take advantage of this. White-Guard is implemented in the "bigger brain" back here in the lab. Please let us know if there is more we can do. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Slow processing times, errors
On 2013-06-28 16:49, Matt wrote: maybe the updates will cause a service restart/reset? Rulebase updates (all updates in fact) happen without restarting anything. SNF simply loads the new configuration, validates it, uses it for new scans, and when all of the old scans are complete it drops the old data. All of this happens without impacting scan operations. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Slow processing times, errors
On 2013-06-28 12:10, Matt wrote: I am looking to retool presently just because it's time. So if you are convinced that this is due to low resources, don't concern yourself with it. Ok. It makes sense that the ~200 messages all at once could have happend at the restart. SNFClient will keep trying for 30-90 seconds before it gives up and spits out it's error file. That's where your delays are coming from. SNF itself was clocking only about 100-800ms for all of the scans. The error result you report is exactly the one sent by SNF -- that it was unable to open the file. I am very sure this is resource related -- your scans should not be taking the amount of time they are and I suspect most of that time is eaten up trying to get to the files. The occasional errors of the same time are a good hint that IO is to blame. The new spam that we've seen often includes large messages -- so that's going to put a higher load on IO resources -- I'll bet that the increased volume and large message sizes are pushing IO over the edge or at least very close to it. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Slow processing times, errors
On 2013-06-27 20:01, Matt wrote:
I'm
attaching a snippet of my log. About 100 lines past the start,
where you see a smattering of error messages, you then see a large
block of them while the Sniffer service is restarting, and then
after that no errors at all. There have in fact been no errors at
all in several hours since this restart of Sniffer.
I can promise you that the error you're reporting comes directly
from a problem with the file system. Here is the code where that
error is generated. Put simply the code tries to open the file and
determine it's size. If that doesn't work it throws the
ERROR_MSG_FILE exception in one of two forms -- that's what ends up
in the log.
try { // Try opening the message file.
MessageFile.open(MessageFilePath.c_str(), ios::in | ios::binary); // Open the file, binary mode.
MessageFile.seekg(0, ios::end); // Find the end of the file,
MessageFileSize = MessageFile.tellg(); // read that position as the size,
MessageFile.seekg(0, ios::beg); // then go back to the beginning.
MyScanData.ScanSize = MessageFileSize; // Capture the message file size.
}
catch(...) {// Trouble? Throw FileError.
MyRulebase->MyLOGmgr.logThisError( // Log the error.
MyScanData, "scanMessageFile().open",
snf_ERROR_MSG_FILE, "ERROR_MSG_FILE"
);
throw FileError("snf_EngineHandler::scanMessageFile() Open/Seek");
}
if(0 >= MessageFileSize) { // Handle zero length files.
MessageFile.close();// No need to keep this open.
MyRulebase->MyLOGmgr.logThisError( // Log the error.
MyScanData, "scanMessageFile().isFileEmpty?",
snf_ERROR_MSG_FILE, "ERROR_MSG_FILE"
);
throw FileError("snf_EngineHandler::scanMessageFile() FileEmpty!");
}
Another clue is that in the log snippet you provide, there are hints
of a problem brewing when there are sporadic instances of this
error. Then, when there is a large block -- virtually all requests
to open the files for scan are rejected by the OS. Either something
made those files unavailable, or the OS was unable to handle the
request. I find it interesting also that the time required to report
the error started at about 172 milliseconds and continued to climb
to 406, 578, and then 656 before the restart.
SNF does not make log entries in the classic log during a restart,
by the way.
Note also the timestamps associated with these events and you can
see that the event was precipitated by a dramatic rise in message
rates. The first part of your log seems to indicate about 7-10
messages per second. During the large block of errors, the message
rate appears to have been in excess of 120 (I counted approximately
126 at timestamp 20130627183819). That's an increase at least an
order of magnitude higher than the rate that was causing sporadic
errors.
I suspect based on the data you have provided that something on your
system generated a very large spike of activity that your IO
subsystem was unable to manage and this caused snf scans to fail
because snf was unable to open the files it was asked to scan.
Your restart of SNF apparently coincided with the event, but since
all of the SMD file names are unique during the event, and since SNF
has no way to generate scan requests on it's own, SNF does not
appear to have been the cause of the event in any way. It was able
to record the event, none the less.
So the question in my mind now is:
* Is there a way to improve your IO subsystem so that it can gain
some headroom above 10 msg/sec?
* What caused the sudden dramatic spike that led to this episode?
On most tiny systems I monitor, scan times are generally < 100
ms. On your system they are frequently in excess of 400 ms which
leads me to believe your system is a bit underpowered for it's
current load.
Hope this helps,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing li
[sniffer] Re: Slow processing times, errors
On 2013-06-27 16:04, Matt wrote: like this: 20130627155608, arg1=F:\\proc\work\D6063018a2550.smd : Could Not Connect! That is SNFClient giving up after waiting for SNF to process the message for too long. At the same time, my Sniffer logs start showing frequent "ERROR_MSG_FILE" results on about 1/8th of the messages. This is SNFServer giving up after trying to open the message file and read it. What's happening is that the OS is not delivering the file to SNF, SNF is waiting for this (it has no choice, it's a call to the OS's open() command), and then eventually it fails so SNF produces the ERROR_MSG_FILE result because it was not able to open the file it was supposed to scan. This is often caused by fragmentation, or it can be that there are too many files in the directory that contains the message file. Ultimately it is an IO problem. This shouldn't be associated with updates -- but if it is, I might guess that's because the file system is ready to fall over and saving a new rulebase file to disk, or reading afterward is enough to push it over the edge. Seeing ERROR_MSG_FILE on 1/8th of the scans means that SNF is being asked to scan a message that the file system can't or won't give it. That is a strong indication that the system is IO bound. SNF can't really do anything different in that case -- it's simply asking to open the file so it can read it. If the IO system says "No" then it spits out that error. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Slow processing times, errors
On 2013-06-27 17:25, Darin Cox wrote: When we had sluggish performance similar that yours, resulting in numerous sniffer .tmp files in the spool, the cause was eventually traced to a proliferation of files in the sniffer directory. Clearing them out brought performance back up to normal. This is usually the problem. NTFS performs very badly when there are a lot of files in a directory -- and that slows everything down. If SNF takes 30 seconds or more to process a message then SNFClient will give up and let the message through (fail safe). _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: 2nd level IP scanning
On 2013-06-07 19:18, Peer-to-Peer (Spam-Filter.com) wrote: I’m seeing one spammer who’s IP address is (x.x.x.x). Or at least that’s the originating IP in the headers. I’m seeing thousands of messages originating from this IP, however they are passing thru hundreds of different Verizon and Comcast servers. Literally. I can’t block (or I don’t want to block) the Verizon or Comcast IP’s, but I need to block the originating IP (x.x.x.x). I think you have misunderstood what drilldown does. If you teach drilldown to recognize the versizon and comcast servers then it will learn to ignore them and pinpoint this specific IP. It will also learn to find any other IPs that are doing the same kind of thing. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: 2nd level IP scanning GBUdbIgnoreList.txt file
On 2013-06-07 18:36, Peer-to-Peer
(Spam-Filter.com) wrote:
Pete:
I’m not sure that GBUdbIgnoreList.txt file would work for my
situation as it seems I would need to trust all IP’s from
Comcast and Verizon to catch this one IP below– Correct? Or am
I misunderstanding.
Perhaps I misunderstand -- but:
I didn't intend to recommend GBUdbIgnoreList.
I DID intend to recommend using drillown directives.
I interpreted your message to indicate there are intermediate
Verizon and Comcast servers you want to ignore when looking for
the source IP.
If I'm right about the intermediate servers then I presume they
would always be intermediate. So, what we want to do is tell GBUdb
to recognize those servers and ignore them. Then it will find the
next received header behind those and treat it like the source.
The process is loosely described here (copied from the page I
recommended):
Suppose some large
ISP uses the domain mixed-source.net, then you might see
received headers similar to:
Received: from out56.mixed-source.net [12.34.56.78] by my0wn1.bestfilterever.net
(1.2.3.4 / 5.6.7.8) so forth and so on;
Received: from inside34.mixed-source.net [210.1.2.34] by outside56.mixed-source.net
(and so forth) and so on;
Received: from border124.mixed-source.net [210.1.2.124] by inside34.mixed-source.net
(and so forth) and so on;
Received: from ugly-spambot-customer.dyn-dsl123.eviltown.cpe9.example.com [99.88.77.66] by
border124.mixed-source.net (and so forth) and so on;
Then your
section might look like this:
The top received
header (ordinal 0) created by your system would match the
first drilldown header directive and so 12.34.57.78 would be
added to GBUdb with the Ignore flag set. SNF would keep
looking.
The next received
header (ordinal 1) created by mixed-source would match the
second drilldown header directive and so 210.1.2.34 would be
added to GBUdb with the Ignore flag set. SNF would keep
looking.
The next received
header (ordinal 2) created by mixed-source would match the
third drilldown header directive and so 210.1.2.124 would be
added to GBUdb with the Ignore flag set. SNF would keep
looking.
Finally, the next
received header would not match any header directives. The
previous received headers have all been made ineligible as
message sources. As a result the IP 99.88.77.66 will be
treated as the source IP for this message.
Ultimately that results in intermediate servers being ignored
always and specifically (by IP) presuming that the actual source
of the message will always be something behind them.
This doesn't trust whitelist those servers -- it simply says we
can't treat them as the message source.
Let me know if I missed something.
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: 2nd level IP scanning
On 2013-06-07 18:16, Peer-to-Peer
(Spam-Filter.com) wrote:
Hey Pete and all,
Is there an option to have SNF scan second or
third deep header IP’s? I’m trying to block an originating IP
(66.83.88.42), however they are hopping thru Comcast and
Verizon.
Yes! You can use drilldown directives to teach SNF to "trust"
intermediate servers and find the originator:
http://www.armresearch.com/support/articles/software/snfServer/config/node/gbudb/training/drilldown.jsp
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-05-24 08:38, Richard Stupek wrote: Pete I thought the local gbudb got updates from the service or was that a future enhancement? That's true right now. GBUdb is part of a distributed machine learning system. There is a conversation going on between all SNF nodes where they share their point of view on IP reputations. This happens approximately once per minute, out of band. Each node alerts the system that they have new activity on a given IP. Then, via the SYNC server(s), each node receives a reflection of the consensus on that IP. So, when an IP is new to a node it will be updated within about a minute with the consensus reputation from the other nodes. As there are more interactions, the consensus matters less and the local experiences matter more -- but the conversation continues so the each node is always influencing the other nodes about any active IPs. The conversation protocols are intelligent so that there is just enough traffic to accomplish the learning goals and so that a hostile / compromised node cannot poison the system; and so that each node can maintain it's own point of view about each IP. For example: Say node A regularly corresponds with an ISP in blackhatistan. So, node A sees a mixture of good and bad messages. Node B only gets bad messages from the same ISP. Node A will have a local reputation for the ISP that is good enough to let messages through on that system, but node B will have a local reputation for the ISP that blocks most messages. The consensus of all GBUdb nodes will be somewhere in between. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-05-23 17:37, Richard Stupek wrote: Mine can speak XCI, its custom. Kewl -- then you can use GBUdb for local IP reputation data whenever you like. That could be very useful. Anything you can do with SNFClient you can do with XCI -- SNFClient is just a command line translator. Since you can do that, one thing you might consider doing is to use GBUdb for targeted gray-listing. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-05-23 17:21, Richard Stupek wrote: Would this: http://armresearch.com/support/articles/software/snfServer/xci/gbudb.jsp yield the same results as using the ip4 blocklist? No. Asking your local GBUdb about an IP will only give you a local perspective. The truncate blacklist contains the currently active worst-of-the-worst as seen by all SNF nodes working together. Also -- getting your MTA to pay attention to your local GBUdb is nontrivial since no MTA software (that I know of) can "speak" XCI yet. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-05-23 16:41, Richard Stupek wrote: Can you point me at the documentation for the truncate blacklist and its usage? http://gbudb.com/truncate/index.jsp It's an ordinary ip4 dnsbl. Most email systems have some mechanism for blocking connections based on this kind of blacklist. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-05-23 15:22, Richard Stupek wrote: Looks like I have this issue again (pegging 4 core cpu) and resetting the process doesn't make a difference. Not sure what is causing it but it does slow down spam detection to 40-50 seconds for many emails. Any ideas what I can look at or do to resolve this? Check the message sizes. As part of the newest spam storms we've noticed that a lot of the messages are huge (65536++). I suspect this might impact throughput as large buffers are allocated and moved around to handle these messages. This kind of thing has also been known to cause NTFS to crawl. Please let us know what you find. If you are not already doing it -- you should consider blocking connections using the truncate blacklist. No sense taking on some of these messages if they can be eliminated up front. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Rulebase Compiler Improvements
Hello Sniffer Folks, We have improved our rulebase compiler scheduling and efficiency. This has allowed us to increase the pace of rulebase updates by approximately 20%. You should see a further reduction in leakage rates and slightly more frequent rulebase updates. Also: Did you know that Message Sniffer is designed not only to filter spam, but also to filter any email containing malware or viruses? We recently performed a spot check by throwing a corpus of 200K known virus emails at SNF. Only 76 leaked. According to that and other similar tests SNF typically captures 99.962% of infected email. This means that for most systems using SNF there is no need to also scan for viruses on the mail server -- SNF does it all at once. Anything that does get by SNF should be captured by desktop virus protection -- something nobody should do without since email is not the only way viruses can get into your systems. It's important to note that infected messages may not be marked as malware -- SNF is designed learn and look for all kinds of unusual artifacts that help to identify unwanted messages and many of those are used not only in malware but also in other kinds of spam. So, a lot of the time infected messages are captured by patterns that were learned while looking at ordinary spam. Please let us know if there is more we can do. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Volume
On 2013-04-26 13:12, Peer-to-Peer (Spam-Filter.com) wrote: Anyone else seeing the same? The spamtrap pre-filter volume is about 4x typical. It's really quite something. The stock-push stuff has a lot to do with it -- since that's become popular again we've seen striking volumes associated with it. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Reputation Lookup DNSBL?
On 2013-04-18 03:06, Andy Schmidt wrote: Is there a GBUdb IP based lookup that is recommended to get the benefit of all Sniffer customers' experiences? There is the truncate blacklist http://www.gbudb.com/truncate/index.jsp Other than that SNF will automatically learn what the other SNF nodes know about an IP within about a minute of the first encounter. Then as your SNF node has more experience with the IP it will begin to trust it's own data more than that of the other nodes. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Upgrading Stand-Alone Sniffer (for Declude)
On 2013-04-18 02:52, Andy Schmidt wrote: SNFMulti Engine Version 3.0.11 Build: Aug 21 2009 18:42:53 SNF Server Version 3.0.2 Build: Jul 28 2009 14:48:00 That is currently the latest official release. There is a slightly newer SNFServer.exe that is an interim (snapshot) release: http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.0.23.zip Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IPScan results
On 2013-04-16 20:40, Peer-to-Peer (Spam-Filter.com) wrote: Once I installed a new Security Plus license, Outbreak Protection engine started and so did SNF IPScan. Around 20120905 I asked about this and Arvel put in a patch to remove the restriction. That is, since Md 13.0, you do not need the security plus license to use the full API on plugins, so the original SNF4MDaemon plugin design will work. (that's what you have configured). Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IPScan results
On 2013-04-16 17:02, Peer-to-Peer (Spam-Filter.com) wrote: Hey all, I noticed a couple of my MDaemon mailservers are not performing the “SNF IPScan”. Check that your MDaemon versions are the same -- some versions implement the plugin API differently. Then make sure that your Plugins.Dat file configures the SNF plugin correctly. If you've got one server working correctly and other's not, then that gives you a good way to compare. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Convert your Declude OEM license now and get full credit!
On 2013-04-11 08:22, e...@insight.rr.com wrote: Because of this entire issue with declude. It might be nice if you contacted smarterTools and offered to work with them on them integrating message sniffer directly into smarterMail. :) We have attempted this on several occasions and it hasn't worked out. We remain optimistic and ready to work with them if they decide to change their minds about it. I know that SNF's features and speed, if fully integrated, would take SM spam and malware filtering to the next level. In the mean time, it is possible to integrate SNF directly with Smarter Mail by calling it as a command line scanner. Then the injected headers can be used in filtering rules or to add weight to the built-in SpamAssassin scores. http://www.armresearch.com/support/qa/integration/smarterMail.jsp Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Convert your Declude OEM license now and get full credit!
Hi Sniffer Folks, It appears that Declude (the company) is failing. After many rumors of problems and some first hand experience, today the Declude web site has gone dark. We have a long standing relationship with the Declude community, and we want to make sure we do what we can to support them even if Declude itself goes away. Place a new order for Message Sniffer (SNF) now and we will give you credit for any time you have left on your Declude OEM license. Tell us your OEM expiration date with Declude and we will add the time you have left to your new SNF license. For the best pricing we recommend you purchase through one of our resellers: https://www.armresearch.com/products/resellers.jsp Please be sure to pass this information on to any interested folks that might not be on this list! There is bound to be a lot of turmoil right now and we don't want anybody to miss it. Please let us know if there is more we can do! Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-03-29 12:59, Richard Stupek wrote: well when all else fails restarting snf seems to have corrected the issue for now. In that case, it is likely that RAM fragmentation was involved. Dropping the process allowed the fragmentation to be cleared. (theory). Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-03-28 12:10, Richard Stupek wrote: Ok looking at the log I see quite a few messages taking over a second to process (samples below): Great! Now we're getting somewhere. It seems likely that your system is bound in 2 ways: * Allocating RAM * Reading / writing from the hard drive. The setup time s='1172' indicates that it took more than a second to allocate a 72K buffer and read the message. That is the only work done during setup. The scan time t='1109' indicates the amount of time it took to complete the rest of the process. I'm guessing that since the setup took more than a second that writing the message back with injected headers probably took a while too. A scan depth of 127 is nominal. The data you sent indicates this is a problem when the messages are fairly large. There is likely a boundary condition between smaller messages and larger ones that allow the smaller messages to be handled more efficiently. Since you are indicating that CPU utilization is high during these events and since you're not mentioning other performance issues, it seems likely that RAM fragmentation or RAM starvation might be the problem here. During the setup, SNF allocates a buffer large enough for the entire message and then reads it all at once. This is most efficient because there is a single system call for each of these events - so the OS has complete control over the entire step and it only happens once. After the scanning process is done, SNF will typically allocate another buffer large enough to include the message and all of it's headers. This new version of the message is constructed in the buffer and then written all at once to the file system. If the problem is RAM fragmentation or starvation then it could be taking as much as a second for the OS to allocate a 72K buffer --- that's the kind of thing that should be nearly undetectable, but in these cases where high CPU loads are reported with this kind of log data I have seen that happen. If the problem were simply IO then it is likely that CPU utilization would be low while the CPUs wait for the IO operations to happen. Of course it could be a combination of things -- it's hard to tell what's happening in the internals of the OS. Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-03-27 17:16, Richard Stupek wrote: The spikes aren't as prolonged at the present. Interesting. A short spike like that might be expected if the message was longer than usual, but on average SNF should be very light-weight. One thing you can check is the performance data in your logs. That will show how much time in cpu milleseconds it is taking for each scan and how long the scans are in bytes. This might shed some light. http://www.armresearch.com/support/articles/software/snfServer/logFiles/activityLogs.jsp Look for something like in each scan. From the documentation: - Scan Performance Monitoring (performance='yes') p:s = Setup time in milliseconds p:t = Scan time in milliseconds p:l = Scan length in bytes p:d = Scan depth (peak evaluator count) Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-03-27 16:49, Richard Stupek wrote: Its odd because the number of messags snf is processing isn't more than usual and the % of spam being detected through snf is actually lower than typical yet is is routinely maxing out 4 processors at 100%. You're saying that SNF is maxing out 4 processors? ... or is the combination of operations on your server maxing out 4 processors? We're using the same engine and ruelbase in our CGP server and humming along nicely at between 2000 - 8000 msg/minute with nominal CPU loads. I don't see anything unusual in your telemetry and I haven't heard any other complaints, so I can't explain why SNF would act differently on your system. I hate a mystery though -- so I would love to get to the bottom of it. Do you see anything else that might be causing the CPU load? _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: IP Change on rulebase delivery system
On 2013-03-27 14:38, Darin Cox wrote: Probably unrelated... and due to a significant increase in spam over the past few days. I agree with that -- our inbound spamtrap pre-processor has seen 4x increase over the past few days so that's likely to be related. Also, Richard, I took a quick look at your telemetry and verified that your rulebase file(s) are up to date. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] IP Change on rulebase delivery system
Hi Sniffer Folks, We are about to change the IP of the rulebase delivery system. This change should be completely transparent and you should not need to take any action; however if you do notice anything unusual please let us know. Thanks, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: MDaemon AV 4.1.5 / SNF plugins Update Warning
On 2012-12-05 08:35, Peer-to-Peer (Spam-Filter.com) wrote: One thing to note: This only occurred on servers where I used SNF_CS_Installer when originally setting up SNF. The Servers where I manually added SNF were not affected. That seems very weird -- how would it know the difference? Any ideas? _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: GBUdb Tool
On 2012-11-27 17:12, Darin Cox wrote: Hi Pete, Would you mind sharing your calculations of confidence and probability? Here is a page on the math: http://www.armresearch.com/support/articles/technology/GBUdb/learns.jsp I'm looking at the stats for p=1.0 and curious about the low confidence values. I would have expected high confidence where there were no good samples and a lot of bad... or do I have something backwards? Confidence is a measure of the number of samples seen. So, if you have only one sample, and it was a bad message, then you have a 100% probability that you will get a bad scan (spam) as far as you know... BUT since you only have one sample, you don't know very much so your confidence is low. If, on the other hand, you have seen a few dozen messages from an IP and all of them were bad then you would have much more confidence in your probability figure. Also, while it's easy to parse, it might be nice if the output had one delimiter between fields instead of being both tab and comma delimited. Makes importing into a database for analysis much easier. I will look into making a different output mode that's easier to parse. The existing one is supposed to be human friendly. Thanks! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] SNFServer Interim Release E3.0.23
Hello Sniffer Folks, Here is a link to an interim release of SNFServer for Win* boxen: http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.0.23.zip This interim release fixes a bug in the previous E3.0.19 interim where large messages might be corrupted during message header injection. This new version has been tested thoroughly against large messages. If you don't recall, the E3.0.19 interim and above allows for up to 8 messages to be scanned simultaneously when sufficient CPU cores are available. If you are running *nix and would like to try the interim version then feel free to pull down the updated SNFMulti source code from the SVN server: https://svn.microneil.com/websvn/filedetails.php?repname=SNFMulti&path=%2Ftrunk%2FSNFMulti.cpp It is not necessary to upgrade your SNF installation if you are not running one of the interim releases. If you are running a production release then you're good to go as you are. Please let us know if there is more we can do. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] GBUdb Tool
Hello Sniffer Folks, We have been playing with a new utility that some of you may enjoy. http://www.armresearch.com/message-sniffer/download/GBUDBTool-V0.1.zip GBUDB Tool allows you to create a list of IP addresses from your GBUdb snapshots (.gbx files). You can select IPs that are "blacker" or "whiter" than a provided probability figure and confidence figure. It outputs one IP per line, optionally with details about the statistics for the IP. This can be useful for feeding-forward blacklists to block at your firewall or for other research purposes. Run GBUDBTool without any parameters and it will tell you about it's command line options. Please let us know if there is more we can do. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] High Throughput Windows version of SNFServer available
Hi Sniffer Folks, If you have a high volume Windows based mail server and you would like to beta test some performance features then you can find a new version SNFServer here: http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.0.19.zip There are two key features we're testing on this version: * If header injection is turned off then the message file will only be read up to a maximum of 32K (or the current scan horizon). * There are now 8 processing channels in the XCI server so that 8 simultaneous scans can occur if sufficient cores are available. If you decide to test this then please let us know. Thanks! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Creeping higher on those rule numbers
On 6/26/2012 9:41 PM, Colbeck, Andrew wrote: Rule number 5 million rolled on by this week. Message Sniffer Rule # 500 was coded by Andy (Worm Thunder) 20120626.1408 SortMonsters Rock! I wonder who won the pool? _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Creeping higher on those rule numbers
On 6/26/2012 9:41 PM, Colbeck, Andrew wrote: Rule number 5 million rolled on by this week. Yes indeed! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 twitter/codedweller # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: FPs on Sniffer-Schemes
On 3/13/2012 11:19 AM, Scott Fosseen [Prairie Lakes AEA] wrote:
Can you check to see if all looks ok with
my copy as well.
Sure. I'll respond off-list
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: FPs on Sniffer-Schemes
On 3/12/2012 5:41 PM, Darin Cox wrote:
Started getting hits at 4:30pm EST up to
15 minutes ago (5:25pm EST).
I think I can see part of the problem (possibly).
I do not have telemetry from your system (based on looking up your
Id from your domain). I suspect this means that you are running an
older version of SNF. By extension, that would mean a couple of
things:
* Your rulebase update would not come as quickly as for most
systems.
* Your SNF engine won't match on many of the newer rules.
* Your SNF engine will not have GBUdb and also will not be able to
auto-panic new rules that conflict with IP reputation data.
Am I right about these assumptions?
If not, then we should figure out why I don't see your telemetry.
Thanks,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: FPs on Sniffer-Schemes
On 3/12/2012 5:41 PM, Darin Cox wrote:
Started getting hits at 4:30pm EST up to
15 minutes ago (5:25pm EST). Not sure if the rule has been
pulled or corrected yet.
It was corrected nearly as soon as it was created. It did escape
into some rulebases - we saw that on our conflict instrument. Most
systems auto-panicked the rule right away. It no longer appears on
our conflict instruments - so there is no reason you should see any
hits from it.
I'm chasing things down to see what I can see -- based on your
message.
Best,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: FPs on Sniffer-Schemes
On 3/12/2012 5:17 PM, Darin Cox wrote:
Hi Pete,
We're seeing a ton of FPs on a
Sniffer-Schemes rule # 4764784.
That rule was detected as an error and removed almost immediately
after it was created.
You should not be seeing any additional hits on that rule.
Best,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Bad rule event
Hi Sniffer Folks, We had a short bad rule event this morning. The following rule IDs were matching unintended text, they were discovered quickly on our conflict instrumentation and removed after approximately 30 minutes. Most systems were rejecting the rules (that's how the conflict instrument gets it's data). 4711347 4711362 4711345 4711346 4711360 4711361 We have identified how the rules were coded and adjusted our practices to make this less likely in future. Also, our system will remember these rules automatically so that we cannot make the same mistake again. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] System Upgrades
Hello Sniffer Folks, Just a quick note to let you know: * We have boosted our rulebase production system. New rulebase updates will arrive about 25% faster on average. * We have optimize Rulebot productivity to respond to a wider range of spam / malware variants automatically. * We have augmented our QC processes to seek out more potential false positive cases and stop them before they occur. * We have added additional research channels to help identify more threats more quickly. --- Note that over the next few weeks we will be making additional changes to our technical infrastructure. During service windows occurring at times of low-activity there may be short disruptions in SYNC server connections and/or rulebase delivery. We will do our best to avoid these, and those that do occur should go unnoticed. Your Message Sniffer software installation is designed for high performance and high availability. It will continue to function normally even if we have a disruption during our upgrades, and it will automatically recover from any such disruption without any assistance. Please let us know if there is more we can do. Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Ok, I'm the 3rd person to ever report the Bad Matrix error on this mailing list
On 1/9/2012 4:15 PM, Colbeck, Andrew wrote:
If there's here for the
SortMonsters, it's to make sure that a "bad matrix"
error doesn't interfere with downloading a fresh
rulebase so that SNFserver.exe can get itself out of
that jam.
SNFServer will reject a bad rulebase and keep running with the old
one. So, if somehow a bad rulebase shows up then SNFServer won't
crash... it will keep trying to get a new rulebase with the
getRulebase script until it is successful. By default it will try
about once every 3 minutes if it's not initially successful.
However, once SNFServer is down for any reason, it will refuse to
start without a good rulebase file to work with.
The SNFClient utility can only attempt to ask SNFServer to scan a
message -- it can't do it by itself. If SNFClient is not able to
contact SNFServer and get a good answer then it will create
SNFClient.exe.err to explain the problem and will return 0 to the
calling program -- It returns 0 as a fail-safe so that the messages
will go through. Better to allow all messages through than to block
any good messages by mistake.
On a well functioning system there should [almost] never be an
SNFClient.exe.err file. I say [almost] because things happen from
time to time on every system.
However, if you see a .err message, check it out. If they persist -
something is wrong.
If you try to start SNFServer and it is unhappy, then download a
fresh rulebase first. It's usually a good quick-fix.
Best,
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: Training GBUdb on the client IP for aol.com
On 10/24/2011 3:47 PM, Colbeck, Andrew wrote: C:\MessageSniffer>SNFClient.exe -test 92.231.217.255 Ok, you're working with a different message here (different IP). If you turn on GBUdb data logging then it will tell you what IP it beleived to be the source. http://www.armresearch.com/support/articles/software/snfServer/config/node/logs/scan/xml.jsp http://www.armresearch.com/support/articles/software/snfServer/logFiles/activityLogs.jsp#XML example like: Hope this helps, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Training GBUdb on the client IP for telus.net
On 10/24/2011 3:36 PM, Colbeck, Andrew wrote: That's a very interesting question, Pete. Are you saying that the section is used to override the normal hop 0 / ordinal 0 IP address? If so, I didn't realize it, I thought this was an an additional IP address for GBU to examine. Yes. The source header directive essentially says, "If you see X, then expect the source IP to be in header Y and don't look anywhere else" Under normal circumstances SNF will attempt to identify the source IP as the first Received [IP] that it does not ignore. I think the answer is "yes", I don't want to inspect the ISP's outbound gateway, and I do want to inspect the "client IP" that originated the email. Looking at these headers, the X-Telus-Outbound-IP: seems to match the deepest Received header (the original source) so I think this will do what you want. I'm a little thrown by the "Outbound-IP:" bit - seems a strange name for the originator, but in this case it seems to line up with the correct header. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Training GBUdb on the client IP for telus.net
On 10/24/2011 3:20 PM, Colbeck, Andrew wrote:
[sniffer] Re: Training GBUdb on the client IP for telus.net
On 10/24/2011 3:20 PM, Colbeck, Andrew wrote: Which is in the GBUDB/Training/Source section as per: http://www.armresearch.com/support/articles/software/snfServer/config/no de/gbudb/training/source-header.jsp That appears to be correct and appears to have worked correctly. Top Received header would have been picked as source IP (unless you already have it ignored). It appears that you have successfully told SNF to find the source IP in the X-Telus-Outbound-IP: header in this case. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Training GBUdb on the client IP for aol.com
On 10/24/2011 3:21 PM, Colbeck, Andrew wrote: As far as I know that one still works. _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Training GBUdb on the client IP for telus.net
On 10/24/2011 2:46 PM, Colbeck, Andrew wrote: would this snippet in snf_engine.xml I don't see the snippet from snf_engine.xml? _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] SNF Server / Client for *NIX updated - IMPORTANT bug fix included
Hello Sniffer Folks, Tarball snf-server-3.0.13.tar.gz has been posted to the Products Page: http://www.armresearch.com/products/index.jsp Or you can download it directly here: http://www.armresearch.com/message-sniffer/download/snf-server-3.0.13.tar.gz This distribution contains some minor bug fixes and code improvements bringing the SNFMulti Engine up to 3.0.17. IMPORTANT: This distribution also contains a "clean" SNFServer/main.cpp that fixes a random crash bug! The previous distribution snf-server-3.0.12 contained testing code that would intentionally force a crash (seg fault) under some specific load conditions. The testing code would make it appear that SNFServer was crashing at random with crashes being more likely under higher load conditions. This testing code should not have escaped the lab and was not intended for use in production. We have reviewed and revised our publishing procedures to ensure this does not happen again. This new distribution snf-server-3.0.13 does not contain the testing code. This bug was not included in Win* distributions - only snf-server-3.0.11.tar.gz and snf-server-3.0.12.tar.gz included the errant testing code. Thanks! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Bug Report: SNFServer for *nix
Hello Sniffer folks, We have discovered that some testing code escaped into the latest tarball: snf-server-3.0.12.tar.gz This testing code intentionally causes SNFServer to crash (seg fault) under special conditions. This was done so that we could examine the resulting core dump. You may experience this as random crashes, especially during spikes in traffic. We are looking at our procedures to see how this happened. When we have resolve that issue we will publish a new tarball. In the mean time you can correct this problem immediately by replacing your SNFServer/main.cpp file with the one found on our SVN server here: https://svn.microneil.com/websvn/dl.php?repname=SNFServer&path=%2Ftrunk%2FSNFServer%2Fmain.cpp&rev=16&peg=16&; Download the file, rebuild SNFServer, and replace your existing SNFServer.exe. Sorry for the trouble. We will get a new tarball out shortly. Thanks! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Bad Matrix errors
On 8/22/2011 4:04 PM, Peer-to-Peer (Support) wrote:
Hello SNF,
I think something broke. I'm
seeing a lot of "Bad Matrix!" warnings in my logs. Likely started about an
hour ago.
Running MDaemon
mailserver.
I note in your telemetry that you have a new rulebase since then.
Have the errors stopped?
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: Bad Matrix errors
On 8/22/2011 4:04 PM, Peer-to-Peer (Support) wrote:
Hello SNF,
I think something broke. I'm
seeing a lot of "Bad Matrix!" warnings in my logs. Likely started about an
hour ago.
Running MDaemon
mailserver.
Most likely a bad rulebase load. Bad Matrix is a safety event - it
is possible, but extremely unlikely under normal conditions.
_M
--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010
#
This message is sent to you because you are subscribed to
the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to:
To switch to the DIGEST mode, E-mail to
To switch to the INDEX mode, E-mail to
Send administrative queries to
[sniffer] Re: Nice job, sortmonsters!
On 8/8/2011 6:27 PM, Colbeck, Andrew wrote: Time to thwart a spam run from a fresh IP address: less than 18 minutes. Thanks for the shout out! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Change in default settings
On 5/9/2011 4:53 PM, Colbeck, Andrew wrote: Pete, for sample on-off='on' I wrote myself this note... ... Is it still valid? Your sample and my own configuration have: passthrough=no On the balance of it, I suspect my own note is wrong, so it would be nice if you could verify it one way or the other. The passthrough option is for local sampling. We have used it occasionally on our spamtrap processors, but not for some time. Passthrough takes any messages that would have been samples and instead of sending them to the virtual spamtrap network it lets them go through with a specific result code. Presumably the local system would see the special result code and treat the message differently. Please leave passthrough='no' Thanks! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: Change in default settings
On 5/9/2011 3:43 PM, Peer-to-Peer (Support) wrote: Hi Pete, Just double checking: My snf_engine.xml file does not have any 'single quotes' around any numbers or characters. See attached as an example. What you have there in that png is your configuration log -- it is SNF's interpretation of your configuration file. The actual configuration file does use single quotes (unless you changed it). _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Change in default settings
Hello Message Sniffer Folks, We're recommending a change in the default settings for message sniffer in order to improve our response times for new campaigns. The change is small and enhances our virtual spamtrap technology so that we see new spams sooner and with greater sampling coverage. If you locate this block of code in your snf_engine.xml file: passthrough-symbol='0'/> You will notice that your settings are probably slightly different. The changes we would like you to make are: peek-one-in='3' grab-one-in='3' Your current settings most likely use higher numbers for these settings. Once you make the change and save your file then Message Sniffer should pick up the changes right away - you do not need to restart Message Sniffer when making adjustments to your configuration. Please let us know if you have any questions. Thanks! _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
