[sniffer] Re: Our IP got listed on GBUdb Truncate

2018-11-02 Thread Pete McNeil
On 11/2/18 11:52, Daniel Bayerdorffer wrote:
>
> Is there anyway for us to see what the offending email was that got us
> on the list? Or some other data point to help us clean up our system?

SNF doesn't leak message info -- With the exception of auto-sampling of
spam (truncated messages, and only if you have it enabled) we don't see
message content. What we do get are anonymous statistics and training data.

The good news is that you are running SNF, so you can scan your messages
and identify any content that might have triggered SNF.

Truncate is trained by counting good and bad events -- bad events are
when a message matches spam/malware patterns.

... so you can actually check with your own scanner.

Truncate is completely automated... so we can't change the list data. It
actually doesn't come from a database but rather by skimming the
telemetry for these events. In effect the reputation for any given IP
resides in each SNF instance around the globe and the truncate list
works by eves-dropping on the conversations between those nodes as they
"discuss" IP reputations.

If the IP is still listed and you send a note to support with the IP
requesting a trace then we can collect some events with timestamps. That
may help you track things down -- but since you're an SNF user you would
probably do better with your own scanner.

Hope this helps.

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list .
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: 
To switch to the DIGEST mode, E-mail to 
To switch to the INDEX mode, E-mail to 
Send administrative queries to  



[sniffer] Happy Holidays!

2017-12-23 Thread Pete McNeil

This is just a quick note to let you all know that we're thinking of you.

On behalf of the whole team:

    We wish you a Merry Christmas and a happy, prosperous New Year.

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Rulebase refactoring

2017-09-07 Thread Pete McNeil

Hi Message Sniffer folks,

Over the past few days we've refactored the databases we use to manage 
our rulebase.


As of about 1600e you should notice that all of the rule IDs in your 
system are significantly smaller and completely different.


Unfortunately, during the transition there were several unforeseen 
problems that introduced delays and other disruptions.


We apologize for the inconvenience.

All is well now.

Thanks,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Reminder - the Rule Panic feature

2017-06-01 Thread Pete McNeil

Hello Sniffer Folks,

In light of today's bad rule event I've discovered that many of you are 
not aware of the rule-panic feature.


The rule panic feature has been built in to the Message Sniffer engine 
for many years now, and I suppose is used so rarely that folks have 
forgotten about it.


The feature allows you to render any single rule inert immediately 
without disrupting anything else in the system. So, it could have been 
used to mitigate this event without taking more drastic measures.


Here is a link to the QA article about the rule panic feature:

http://www.armresearch.com/Documentation/QA/ltrulepanicsgt-628138610.jsp

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Bad Rule Alert 2654821

2017-06-01 Thread Pete McNeil

Hello Message Sniffer folks,

This morning a dormant rule from 2009 was reactivated when new messages 
reached our spamtraps this morning matching the rule.


Unfortunately rule 2654821 causes a high rate of false positives in our 
current year that it apparently did not cause back in 2009.


Since the rule was not recently coded and had been in the system for so 
many years our monitoring systems did not immediately detect the rule as 
a false positive case.


However, the team did discover the problem after a few hours and removed 
the rule.


This is the only time an old, reactivated rule has caused significant 
false positive cases -- so it is an exceedingly rare event. None the 
less we are in the process of reviewing our tools and processes to 
improve our sensitivity should any similar event occur in the future.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: rule panic not working

2016-12-29 Thread Pete McNeil

  
  
On 12/29/2016 08:55 AM, Daniel Ivey
  wrote:

Thanks,
but
it appears that my server is failing multiple 54- rules.  For example from Google,
it is failing 54-8064853-304-318-m
and 54-8064853-0-2423-f while from Yahoo it is failing
54-8064853-2063-2077-m
and 54-8064853-0-3703-f.

That is in fact a single rule hitting in multiple places.

http://www.armresearch.com/Documentation/QA/ltmatchesgt-1193870513.jsp

The rule ID is 8064853.
The rule has been removed.
_M


-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list <sniffer@sortmonster.com>.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>

To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>

To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>

Send administrative queries to  <sniffer-requ...@sortmonster.com>




[sniffer] Re: Error Code 69

2016-12-15 Thread Pete McNeil

On 12/15/2016 07:04 AM, Don Winsauer wrote:

I have had 419 occurrences of this error since the 1st of the month.  I don't 
even run a virus scanner on this Windows mail server.  We are running IMail, 
Declude with Sniffer.


This could be an indication of a file system problem?

The only reason that occurs is when the file system / OS prevents SNF 
from removing the original file.


Are the files still there?

What changed since the 1st? (did the problem begin then precisely?)

_M


--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: Error Code 69

2016-12-14 Thread Pete McNeil

On 12/14/2016 06:12 PM, John Tolmachoff wrote:




When SNF is configured to inject headers it does so safely---
First, it reads the entire original message into a buffer, then scans it,...
Then it writes a new copy of the message to a .tmp file with the headers 
injected.
When that completes without errors, it deletes the original and renames 
the .tmp file in it's place.
That way, if anything goes wrong, the original is always there as a 
backup until the last moment.


The error above indicates that SNF was trying to delete the original 
message file so that it could move the new version over. Something went 
wrong and it wasn't able to do that.


On windows systems this is most likely because something removed the 
file first -- perhaps a virus scanner or some other process.


On linux systems it's usually a permissions issue (but this does 
sometimes happen on windows in rare cases).


So, if you can figure out what is preventing SNF from deleting the 
original file you will solve the problem.


Hope this helps,

_M


--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: DEB Packages

2016-12-01 Thread Pete McNeil

On 12/01/2016 02:07 PM, Daniel Bayerdorffer wrote:
I see that the DEB packages for Message Sniffer are for Ubuntu 14.04. 
Will these work with 16.04?




They should -- there haven't been any significant changes in SNF nor in 
the parts of Ubuntu that SNF cares about.


Still, the packages are considered experimental (mostly due to a lack of 
exhaustive testing) so be ready to roll back just in case; and do share 
your results with us.


Best,

_M


--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: .smd.tmp files being left in proc\work folders

2016-08-09 Thread Pete McNeil

On 08/09/2016 05:11 PM, Don Winsauer wrote:

These file are being left and not being delivered.  They are usually over 20mg.


Something is preventing SNF from renaming the file. Find out what it is 
and then prevent it from blocking SNF. Perhaps a virus scanner has the 
file open when SNF comes by to rename it??


SNF tries to do everything safely, so when injecting it's headers it 
writes the new message file with a .tmp extension and makes sure that 
succeeded before it removes the original and then renames the .tmp.


My guess is that the new .tmp file catches the attention of some scanner 
or other program and that when SNF goes to rename the .tmp file to 
replace the original it is unable to do it.


Hope this helps,

_M


--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] SNF Engine Update to 3.2.1 / Short Buffer Bug Fix

2016-04-19 Thread Pete McNeil

  
  
Hi Sniffer folks,

Today we have released a new SNF engine with a minor bug fix. Please
update your SNF installation at your convenience. Chances are that
you've not seen any problems from this bug. If you have experienced
problems they most likely presented as very rare, random errors
possibly causing a crash.

As with most SNF engine updates the simplest process is to replace
your binary with the latest. For windows users here are some links
to the latest engine:

http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox32-3.2.1.exe
http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox64-3.2.1.exe

Simply stop your SNFServer, swap in the new .exe (renamed of course)
and restart SNFServer.

For folks running linux platforms the packages and source tarballs
on our web site have all been updated.

OEMs using the windows SDK should upgrade to the latest DLL which
should be a swap-in replacement.

http://www.armresearch.com/Downloads/index.jsp

---

Technical details:

The bug fix is for a short buffer allocation in the
codedweller/configuration.cpp module. The bug fix also solves
problems unrelated to SNF where applications using the
CodeDweller/configuration engine to parse unusually large XML
attributes could cause a stack overflow.

The solution allocates the buffer for attributes from the heap
instead of the stack and eliminates a short-by-one allocation error.

Those curious about the source code can see the important diff here:



Best,

_M
-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  



[sniffer] Re: [Alligate]Alligate and Sniffer again (NL)

2016-01-18 Thread Pete McNeil

  
  
On 01/18/2016 07:26 AM, Bonno Bloksma
  wrote:


  
  
  
  
Hi,
 
Ok, downloaded Alligate trial, installed in on
a 2012 R2 server.
Made a local dns “server” (resolver) on the
machine but I am not sure if I need it now that we can use
the Google dns server by default.
  


A local resolver will speed up your bl lookups quite a bit since
they don't have to traverse the network.


  

 
How do I hook up Sniffer? I used to have
Declude (and IMail) and had Sniffer connected that way, I
now need to connect sniffer into Alligate.
I cannot find anything in the Alligate Docs I
downloaded.
  


As far as I know SNF4Alligate still works.

https://www.armresearch.com/Documentation/Papers/InstallGuides/SNF4Alligate.jsp

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list <sniffer@sortmonster.com>.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>

To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>

To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>

Send administrative queries to  <sniffer-requ...@sortmonster.com>




[sniffer] Re: New Version -- SNFMulti 3.2.0 -- Strangers

2016-01-04 Thread Pete McNeil
On 2016-01-04 11:44, Daniel Bayerdorffer wrote:
> Are there any other gotcha's I should be aware of?
I took a quick look through the tarball and was reminded -- all of the
configuration elements are provided as samples after make-install. The
instructions say to copy the samples to their correct names and then
modify them appropriately-- so that part of it is a manual process. In
that case it should be safe to do make install and just skip those steps
since your configuration is already happy.

All that said; again -- you're really only interested in updating your
SNFServer binary. The rest isn't changed.

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: New Version -- SNFMulti 3.2.0 -- Strangers

2016-01-04 Thread Pete McNeil
On 2016-01-04 11:44, Daniel Bayerdorffer wrote:
> Hi Pete,
>
> I have a couple of questions about upgrading. We will be upgrading SNF4SA 
> running on Ubuntu 14.04 with Zimbra email server.
>
> I previously compiled the source code to install SNF4SA. Can I compile the 
> latest version and run the make-install to overwrite the existing version? If 
> so, do I need to re-apply our license information to the configuration files, 
> etc.?

I think that should work.
The only thing that's really different is SNFServer.
You could stop after make and then replace your SNFServer binary leaving
everything else in place.

I don't think there are any other gotchas.

Best,
_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: [BULKMAILER] [sniffer] Windows SDK with SNFMulti 3.2.0 -- coming soon.

2015-12-29 Thread Pete McNeil
On 2015-12-28 11:28, Michael Murdoch wrote:
> I AM.

The latest Windows SDK is posted. It's exactly like the previous one
except we changed the version number and the DLLs have been updated.
They should be a drop-in replacement for the previous DLLs.

http://www.armresearch.com/message-sniffer/download/updates/SNFMultiSDK_Windows_3.3.zip

Please let us know if there is more we can do.

Best,

_M


-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Windows SDK with SNFMulti 3.2.0 -- coming soon.

2015-12-24 Thread Pete McNeil
Hi Sniffer Foiks,

If you're curious about the Windows SDK (DLLs) ... they should be posted
in the next few days, but not yet.

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] New Version -- SNFMulti 3.2.0 -- Strangers

2015-12-24 Thread Pete McNeil
Hello Sniffer Folks,

A new version of Message Sniffer is available. The most exciting new
feature for this version is: Strangers.

The "Strangers" algorithm replaces the previous White-Guard algorithm.

Strangers prevents high-intensity pre-tested spam from poisoning IP
reputations in GBUdb and enhances SNF's sensitivity to these kinds of
attacks. Once pattern rules begin to match the pre-tested attack the IP
reputations quickly climb into the black enhancing all of SNF's learning
systems. Normal, but new, IP sources are held to low-confidence
reputations for several hours, but after that are allowed to develop
normally.

Short summary: Strangers lets SNF close the door more quickly on
pre-tested spam while enhancing SNF's learning sensitivity to those
events and without interfering with normal IP reputation processing.

Here are some links:

Packages from the LabRats...
http://www.armresearch.com/message-sniffer/download/packages/

SNFMilter tarball...
http://www.armresearch.com/message-sniffer/download/updates/snf-milter-1.2.0.tar.gz

SNFServer tarball...
http://www.armresearch.com/message-sniffer/download/updates/snf-server-3.2.0.tar.gz

SNFServer 32bit Windows exe...
http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox32-3.2.0.exe

Not better, but if you _really_ want it ... SNFServer 64bit Windows exe...
http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox64-3.2.0.exe

Thanks! and Happy Holidays!

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] ShortMatch Resolved - Update your SNF software to remain immune.

2015-12-03 Thread Pete McNeil
Hi Sniffer Folks,

According to our latest data, the Short-Match FP problem has subsided -
most likely due to rule sequestration. We have not seen any significant
events in our detection software since 2100e last evening.

In the mean time we have updated the SNF software to check for
short-match events and treat them as rule-panic events. This renders
them inert so that if this kind of rulebase corruption occurs again the
SNF engine will be immune.

Please update your SNF software to this latest version using the links
below.

NOTE: The Windows installer is in the process of being redesigned and
does not have the latest software. This will take some time. If you are
using SNF on Windows and use(d) the installer then use this procedure to
update your software:

* Stop your SNF service (usually XYNT Service based).
* Copy your SNFServer.exe file to SNFServer.old
* Download SNFServer-windows-7-prox32-3.1.0.exe (32 bit) or 
SNFServer-windows-7-prox64-3.1.0.exe (64 bit) and rename it to
SNFServer.exe to replace your previous SNFServer.exe.
* Start your SNF service.

If you were using the 32 bit version (very likely) then replace it with
the 32 bit version. There really isn't any difference, but just in case
it's simpler to keep things the same. There is no benefit to running the
64 bit version -- It is not faster and is in fact less efficient due to
the use of extra large (64 bit) pointers that aren't necessary ;-) Some
folks really want a 64 bit version, so we have one.

Here are some links to updated versions:

http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox32-3.1.0.exe
http://www.armresearch.com/message-sniffer/download/updates/SNFServer-windows-7-prox64-3.1.0.exe
http://www.armresearch.com/message-sniffer/download/updates/snf-server-3.1.0.tar.gz
http://www.armresearch.com/message-sniffer/download/updates/snf-milter-1.1.1.tar.gz
http://www.armresearch.com/message-sniffer/download/updates/SNFMultiSDK_Windows_3.2.zip

And for the really adventurous:

http://www.armresearch.com/message-sniffer/download/packages/

In the packages link you will find all of the latest snapshots and some
old ones from our LabRats. The LabRats compile and test SNF for all of
the different platforms. You will find RPM and DEB packages as well as
tarballs and even the windows stuff that's posted in the updates links
above. Be sure to pick the latest version in all cases.

It will take a bit of time before all of the ordinary links on our web
site are updated with the latest software, so please use the above links
instead if you're going to update right now.

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: ShortMatch Resolved - Update your SNF software to remain immune.

2015-12-03 Thread Pete McNeil
On 2015-12-03 21:24, Daniel Bayerdorffer wrote:
> Just so I understand correctly, can we use the packages to install over a 
> current installation that was compiled from source?

Probably not -- the deployment might not be exactly the same.

If you originally compiled from source then your easiest solution will
be to use the tarball and compile from source again. Then you can simply
replace the executable you have with the new one you make -- everything
is compatible and nothing will need to move.

If you use the packages you are essentially starting over. The packages
are deployed differently than the source instructions.

For example, to do the generic postfix integration with SNF Server you
would need to install two packages: the snf-server_ package and then the
snf-server-postfix_ integration package. If you wanted to roll your own
integration you might just install the snf-server_ package and then
build your own scripts and other software on top of that. It's a
different paradigm.

Hope this helps,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: Short Match FPs.

2015-12-01 Thread Pete McNeil
On 2015-12-01 18:12, Darin Cox wrote:
> Thanks for the info, Pete.  Appreciate your proactiveness on this.
>  
> Hope you had a good Thanksgiving!

Thanks! I did.

I'd also like to report that some of our experiments might be showing
results.
It is possible that the trouble has been mitigated based on the latest
data I'm seeing.
I will know better how good this data is after about an hour.

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: Short Match FPs.

2015-12-01 Thread Pete McNeil
Hi folks,

Good News!

After much research and experimentation we have determined that some
time on Nov 28 a corrupted rule entered the rulebase and caused the
intermittend short-match problem. We have removed a group of rules
surrounding that timeframe and have observed a 3 sigma drop in the rate
of short-match events. This indicates that the problem is solved and not
likely to return.

Now that we know this kind of event is possible (it's not supposed to be
mathematically) we will be building a detection and mitigation strategy
into the engine... just in case it does happen again. Once in two
decades makes that seem unlikely.

We will also be continuing our research on the sequestered rules to
identify the one(s) that caused the problem and identify a way to
prevent that recurring.

In the mean time the detection mechanisms we used to monitor our
experiments will remain in place so that if we do see any future events
we will be able to identify them much more quickly.

Sorry for the trouble,

Thanks for your patience and continued support!

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: Question, changing from SNF4SA to Milter, using freebsd

2015-09-08 Thread Pete McNeil
On 2015-09-08 04:04, P Pruett wrote:
>
> Interesting, yes, the spamassassin SNF4SA does seem to be able to use
> snf-milter instead of snf-server.

That's probably not a good way to go. This will cause each message to be
scanned twice. Once by the milter and again by the engine via SNF4SA.

If you want to use SNF4SA then you should turn off the milter and use
SNFServer instead.


> On freebsd 9.3 with Sendmail, I did add the milter and restarted sendmail
> and its seems to be playing okay.
>
> Now I turned it on, I am not sure what the snf milter is doing.

That will depend on how you have it configured. The milter interface
only provides a few options. Your SNF log should tell you what was found
in the scan and the snfmilter configuration will tell you what SNF told
the milter to do.

> Can you point me to some more documentation with details about what the
> milter is doing?
> From what I saw in the setup file it can Allow, Accept, Retry, Reject

That is defined by the milter interface.

Milter.org was shut down permanently just recently.
That page says this is where to find documentation on milters:
http://www.sendmail.com/sm/open_source/download/

>
> I was think it might insert information in the header

SNFMilter should inject the usual SNF headers if they are configured
(they are by default).

>
> Would be nice if the milter could be somehow be used to promote IP
> addresses into a pf table
> for the pf firewall  to redirect with?

That's an entirely different software project. If you want that kind of
functionality then you'd do better to use SNFServer/SNFClient in a
postfix filter. The filter script could then be modified to look at the
results and respond in any way you can code.

Best,
_M



-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Re: Question, changing from SNF4SA to Milter, using freebsd

2015-09-06 Thread Pete McNeil
On 2015-09-06 13:11, P Pruett wrote:
> So what "gotchas" do you know that I need to be aware of if I already
> have snf-server
> setup and I am going to try snf-milter?

The two are not designed to work together.

It turns out that SNFMilter has the full SNF engine in it so if you have
SNFMilter running you should also be able to use SNFClient and things
that act like SNFClient such as SNF4SA.

This is not something we test heavily though because almost nobody tries
to do this. Most folks who run SNFMilter either build their own software
to manage messages (Milter API is highly restrictive) or have SNFMilter
inject headers that are later consumed by SpamAssassin and other
ubiquitous tools so that they can customize their system easily.

If you are using SA after SNFMilter, consider simply adding rules that
recognize headers injected by SNFMilter and add appropriate weights for
SNF's results. This is a common and successful configuration which
allows you to reject some messages during SMTP with SNFMilter and then
score the remaining messages using SNF's scan results with SA and other
tools that are usually bundled with SA.

You shouldn't try to run SNFMilter and SNFServer on the same system at
the same time. If you have SNFMilter running, the SNFServer "back-end"
should already be provided in that service. (Check that XCI is on, it
should be by default). In that case running SNFServer would be redundant.

Hope this helps,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list <sniffer@sortmonster.com>.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: <sniffer-...@sortmonster.com>
To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com>
To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com>
Send administrative queries to  <sniffer-requ...@sortmonster.com>



[sniffer] Bad Rule Alert: 6948148

2015-02-23 Thread Pete McNeil
Rule 6948148 was coded as an abstraction to a fake header and was
rapidly removed by QC checks.
Most systems are automatically removing this rule.
The rule coding has been added to our problematic group so that it
cannot be reinvented.
Due to our auto-panic feature it is likely this rule will not affect
most systems.
We apologize for any inconvenience.

Best,
_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Adding Message Sniffer to Zimbra

2015-02-10 Thread Pete McNeil
On 2015-02-10 01:20, Daniel Bayerdorffer wrote:
 But there are no headers in the messages showing snf's results. I can see 
 that the snf4sa.cf has it set to add them though.

 # Header line containing the results from SNFServer.
 add_header all SNF-Result  _SNFRESULTTAG_
 add_header all MessageSniffer-Scan-Result _SNFMESSAGESNIFFERSCANRESULT_
 add_header all MessageSniffer-Rules _SNFMESSAGESNIFFERRULES_
 add_header all GBUdb-Analysis _SNFGBUDBANALYSIS_

 Do you have any more suggestions?

Unfortunately, some implementations of SA are hiding these headers.
We've seen this a few times recently. There doesn't seem to be a way
around it outside of hacking SA itself. (A few people have done that,...
but it was ugly).

If you want to be able to more easily associate SNF logs with messages
you might consider changing SNF's message identifier to use the Message ID.

http://www.armresearch.com/Documentation/QA/ltidentifiergt-2021367617.jsp

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: milter and smtp auth

2015-02-10 Thread Pete McNeil
On 2015-02-10 11:23, Thomas Klaube wrote:
 Sometimes we see false positives from some of the users although
 they have been authenticated correctly. Is there a way to tell 
 SNFMilter to whitelist authenticated users?

There is no such mechanism in Message Sniffer at this time.

I might also point out that white-listing mechanisms generally lead to
abuse. For example, much of the worst malware these days infects a
machine, gain's authentication through email and other systems, and then
uses the authenticated accounts to spread itself further -- this vector
takes advantage of social hacking (trust of known friends/peers) and
hard security hacking (by gaining access to secured accounts the old
fashioned way, by stealing the keys).

We don't get many requests for this kind of thing -- I'm pretty sure
this is the first time I've heard this one.

SNFMilter is distributed as source code so you certainly could code this
modification yourself if you need it for your system, or you might use a
different milter to force acceptance of messages that you've whitelisted
either by list or by behavior.

Please if you do find a false positive do report it to us so that we can
adjust the filters appropriately... much better to get the filtering
right than to make holes in it.

For reference: http://www.armresearch.com/Support/falsePositives.jsp

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: milter and smtp auth

2015-02-10 Thread Pete McNeil
On 2015-02-10 14:53, Thomas Klaube wrote:
 I might also point out that white-listing mechanisms generally lead to
  abuse. 
 I tend to agree that white-listing is usually not the best solution 

 But please consider this case: one of our users tries to relay mail 
 through our servers and is originating from a Dial-up IP address with
 very bad reputation (maybe within truncate) but is correctly authenticated.
 Would you agree that such mails should not be marked as spam or even 
 discarded (at least not based on IP address reputation)?


My answer in this case is - it depends. Some systems I know of would
consider this too high a risk as you've described it. Others would
completely agree that any authenticated system should automatically be
white-listed. Unfortunately for the latter group this often costs them a
lot in clean-up consulting fees when customers get infected. (we see
that a lot lately).

Since this is a policy based decision, you could take advantage of the
GBUdb drilldown feature and teach your SNF to trust the IPs that this
customer might use. What would happen then is that SNF would not be able
to identify the source IP and so only the pattern matching engine would
apply.

http://www.armresearch.com/Documentation/QA/ltdrilldowngt--468945561.jsp

Effectively you'd be telling SNF not to worry about the IP address for
this customer (or for that matter any of the IPs used for dialup by the
customer's provider)... only pay attention to pattern matches.

That's still making a hole,... but it's your hole and you know why you
made it. It's also a pretty small one because if some known spam or
malware comes from there it will still get tagged -- maybe not as
efficiently -- but it will still get tagged.

Hope this helps,
_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Adding Message Sniffer to Zimbra

2015-02-09 Thread Pete McNeil
On 2015-02-09 16:23, Daniel Bayerdorffer wrote:
 libpthread package they have listed for 14.04. But the config script still 
 can't find that library. Can you offer any advice?

apt-get install build-essential

seems to be the equivalent of CentOS

yum groupinstall Development Tools

which usually solves this problem for redhat variants.

Give that a shot and see if it fills in the holes.
Usually by the time I've got g++ up and running on ubuntu it just
works -- hopefully that's not broken in 14.

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Adding Message Sniffer to Zimbra

2015-02-03 Thread Pete McNeil
On 2015-02-02 19:53, Daniel Bayerdorffer wrote:
 Does anyone have any advice or tips for adding Message Sniffer to
 Zimbra 8.6? Specifically with Zimbra's implementation of spam assassin?

The SNF4SA plugin included with the Linux source code distribution
should do the trick. SNF4SA looks to SpamAssassin like any other SA
plugin. It creates a temp file of the message, calls SNFServer to scan
the message, and then processes the results in a way SA expects so it
can be scored.

It _should_ be as easy as that.

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Report one off spams

2014-12-16 Thread Pete McNeil
On 2014-12-16 13:59, John Tolmachoff wrote:
 When sending occasional one off spam not caught to spam@ would it help to 
 attach the original headers and source of the body as text files to the 
 forwarded email?
Not usually -- that would complicate things.

If we can get the original message in it's original form (like
redirecting (not forwarding) from Thunderbird) that would help.

However, if simply forwarding from some other client, the less extra
done, the better it will be. On our end we have to plow through a lot of
different formats and sources, so the simpler everything is the more we
are able to decipher what we're looking at and locate useful artifacts 
structures.

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 


#
This message is sent to you because you are subscribed to
  the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Bad rule report 6237276

2014-03-19 Thread Pete McNeil

Hi Sniffer Folks,

A short time ago Rule 6237276 was detected on our conflict instruments 
and removed from the core rulebase. The rule was in place from 
approximately 1130 to approximately 1400.


We recommend that if you have the ability to release messages matching 
this rule from your quarantines and rescan them then please do so.


The rule was coded to catch a variant of the /goo.gl/ link spam and 
was coded too broadly. The rule was removed when we identified it on our 
IP/Rule conflict instruments and reexamined it. It's signature on the 
conflict instrument is already falling dramatically and we expect that 
most systems auto-panicked the rule making it inert automatically.


We are very sorry for any trouble.

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Saccades anyone?

2014-02-18 Thread Pete McNeil

On 2014-02-18 17:02, Daniel Bayerdorffer wrote:

Any plans to modify the milter code to this in the future?

Yes. All platforms will be updated shortly.
In fact, if you wish, you can download the snfmulti source from our SVN 
server and then recompile your milter with the new code. Here is a link:


Examine it here with websvn
https://svn.microneil.com/websvn/listing.php?repname=SNFMulti

Get the source here via svn
https://svn.microneil.com/svn/SNFMulti/trunk/

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Saccades anyone?

2014-02-13 Thread Pete McNeil

Hello Sniffer Folks,

We are preparing to release a new version of the Message Sniffer engine 
that includes an exciting new technology.


The saccades engine allows SNF to intelligently skip large portions of 
most messages without missing any important content. The engine borrows 
from MicroNeil's synthetic intelligence research relating to visual 
systems processing and essentially gives SNF a behavior similar to what 
we all do with our eyes: http://en.wikipedia.org/wiki/Saccade


The engine learns where matches are most likely to occur and then 
applies what it is learning in real-time. This allows SNF to rapidly 
identify messages of a type it has already seen without having to scan 
the entire contents.


This has the potential to improve scanning efficiency by 90% or more. 
That is, scanning typical messages can happen with 1/10th the work for a 
10x improvement in efficiency. Not kidding, we're actually seeing these 
results on some of our testbed servers! You may have seen me tweet about 
it: https://twitter.com/codedweller/status/434020178352148480


If you'd like to get in on the fun early and you are using SNFServer.exe 
then you can find a copy of the new engine at the following link:


http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.1.0.zip

To swap it in,

* Download and unzip the new engine.
* Stop your Message Sniffer.
* Rename your SNFServer.exe to something like SNFServer.exe.bakup 
(always a good idea to keep a backup).

* Rename the new engine to SNFServer.exe
* Restart your Message Sniffer.

Please let us know how this works for you.

Thanks!
_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: increase in missed spam

2014-02-05 Thread Pete McNeil

  
  
On 2014-02-05 13:56, Herb Guenther
  wrote:

For
  the last week or 10 days I have seen an increase in missed spam in
  Sniffer, Declude seems to be picking it up but I require more than
  a single hit to filter. Anyone else seeing this?

This is what we are seeing.
The trend has been toward very high volume spikes.



To be clear, the graph shows new spam not yet filtered, so the
higher numbers mean higher numbers of new campaigns with higher
diversity.

Hope this helps.

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  



[sniffer] Re: large log.xml files

2014-01-22 Thread Pete McNeil

On 2014-01-22 10:33, Daniel Ivey wrote:

I was checking out our Imail servers this morning and noticed that under the
imail\declude\SNF folder I have a lot of .log.xml files from Sniffer.  Is
there a way to turn off these files in Sniffer or at least to have it only
store about 3 days worth?


If you're not using them you can turn them off:

http://www.armresearch.com/Documentation/QA/ltxmlgt--999318835.jsp


   I also noticed that the size of these files has
grown from about 60 megs a day to over 500 megs the past couple of days.
Does anyone have any ideas as to why the file sizes would increase so much,
I haven't seen an increase in messages.


We have seen a very large increase in the number of messages... that 
might explain it.
Still, that's an order of magnitude there so you should take a look at 
the large files and see if something else is happening.


Best,

_M


--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Bulk / Noisy Rule Group

2014-01-03 Thread Pete McNeil

Hi Sniffer Folks,

Some of you have been experimenting with our Bulk / Noisy rule group 
which is currently tagged with code 65. This above band rule group 
matches anything that might be bulk mail, list mail, etc... similar to a 
popular feature of Postini in the past. As an above band rule group it 
does not train GBUdb, and can only be reliably detected by systems that 
are set up to look for it in SNF's result data.


If you are using it -- then you know you are because you've had to tweak 
your systems to expect it.


This note is to let you know that we will be changing this result code 
to 100 next Friday. The change is to avoid any conflicts with some 
existing error result codes before we make this feature available more 
broadly.


If you are curious about this feature let us know and we will be happy 
to answer any questions you have.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Happy New Year!!

2013-12-31 Thread Pete McNeil

Happy New Year!!

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: What is your oldest production CPU?

2013-12-27 Thread Pete McNeil

On 2013-12-27 15:45, Matt wrote:
Intel 5400 series Xeon here.  But don't forget virtualization.  I'm 
not sure what CPU virtualization does to targeting your code.


That's a good point The processor should be specified in the VM 
profile and if I recall correctly it is typically defaulted to the 
processor of the VM host. I should look closer at this -- but would like 
some feedback.


Thanks,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Whitelist HOW?

2013-11-28 Thread Pete McNeil

On 2013-11-28 19:55, A wrote:
I want to set an ignore list in my MessageSniffer installation, so I 
can receive FBL complains from major ISP.




The directives you've set up will adjust GBUdb training, but SNF pattern 
rules will still tag messages if they match.


Normally if you want to globally white-list a particular sender or a 
particular mailbox on your system you would do that within other systems 
that you use to process the messages -- so, for example, you might add a 
rule to your SpamAssassin configuration to white-list all messages going 
to a particular address; or, if you're using SNF with a postfix 
filtering script then you would create some scripting to ignore SNF scan 
results when that mailbox is the recipient; or if you are using one of 
the many Windows based email platforms then you would enter an 
appropriate processing rule in the MTA to bypass all filtering rules etc...


If you want your SNF rulebase to have a custom white-rule then we can 
code that for you -- send a note to support@ describing the custom rule 
you want coded. Be aware, however, that custom white-rules often have 
unwanted side-effects including that they can be discovered and abused 
by attackers.


Hope this helps,

_M


--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Milter Version

2013-10-31 Thread Pete McNeil

  
  
On 2013-10-31 15:01, Daniel
  Bayerdorffer wrote:

 Ive been reading the Install notes, but
one thing that is not clear is that the Milter version is up to
date. Is it current and if not will it be in the near future?

We have several folks using SNFMilter with postfix et al and no
problems.
As far as I know it's up to date :-)

SNF in general is built to be stable and highly available, so most
of the changes over time happen in the rulebase and not in the
engine.

Hope this helps,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: snf plugin regions question

2013-09-10 Thread Pete McNeil

  
  
On 2013-09-10 11:15, Peer-to-Peer
  (Spam-Filter.com) wrote:


  
  Regarding the regions section in the
SNF Plugin, were currently using the defaults (see below).
  Could you give us a refresher: Why are there
2 entrys for each (white / caution / black)?
  For example:
   edge
probability='-1.0' confidence='0.4'/
   edge
probability='-0.8' confidence='1.0'/
  


They define corner points on a parallelogram that maps the region in
the x,y space:

http://www.armresearch.com/support/articles/software/snfServer/config/node/gbudb/regions/

The points you listed above define the white region with two points.
These points essentially define a line in the graph.
Everything to the left of that line (in the white region) is
considered to be "inside" the region.

Please let us know if there is more we can do.

Best,

_M
    
    -- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: snf plugin regions question

2013-09-10 Thread Pete McNeil

  
  
On 2013-09-10 17:02, Peer-to-Peer
  (Spam-Filter.com) wrote:

Is
that the right direction?

That would open up the black range a bit.
Use caution :-), but have fun.

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] White-Guard

2013-08-26 Thread Pete McNeil

Hi Sniffer Folks,

We've been experimenting with a new machine learning behavior. 
White-Guard is improving early capture rates for new spam and with it 
overall accuracy and throughput. For example, one thing we've seen since 
implementing White-Guard is higher truncate numbers across the network-- 
meaning that more messages are blocked for having bad IP reputations 
than before we implemented White-Guard.


Here is a new blog post that explains what White-Guard is and how it works:

http://www.lifeatwarp9.com/2013/08/lies-machine-learning-and-blackhatzes/

You DO NOT need to install or change anything to take advantage of this.

White-Guard is implemented in the bigger brain back here in the lab.

Please let us know if there is more we can do.

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Slow processing times, errors

2013-06-28 Thread Pete McNeil

  
  
On 2013-06-27 20:01, Matt wrote:

I'm
  attaching a snippet of my log. About 100 lines past the start,
  where you see a smattering of error messages, you then see a large
  block of them while the Sniffer service is restarting, and then
  after that no errors at all. There have in fact been no errors at
  all in several hours since this restart of Sniffer.


I can promise you that the error you're reporting comes directly
from a problem with the file system. Here is the code where that
error is generated. Put simply the code tries to open the file and
determine it's size. If that doesn't work it throws the
ERROR_MSG_FILE exception in one of two forms -- that's what ends up
in the log.

try {   // Try opening the message file.
MessageFile.open(MessageFilePath.c_str(), ios::in | ios::binary);   // Open the file, binary mode.
MessageFile.seekg(0, ios::end); // Find the end of the file,
MessageFileSize = MessageFile.tellg();  // read that position as the size,
MessageFile.seekg(0, ios::beg); // then go back to the beginning.
MyScanData.ScanSize = MessageFileSize;  // Capture the message file size.
}
catch(...) {// Trouble? Throw FileError.
MyRulebase-MyLOGmgr.logThisError(  // Log the error.
  MyScanData, "scanMessageFile().open",
  snf_ERROR_MSG_FILE, "ERROR_MSG_FILE"
);
throw FileError("snf_EngineHandler::scanMessageFile() Open/Seek");
}

if(0 = MessageFileSize) {  // Handle zero length files.
MessageFile.close();// No need to keep this open.
MyRulebase-MyLOGmgr.logThisError(  // Log the error.
  MyScanData, "scanMessageFile().isFileEmpty?",
  snf_ERROR_MSG_FILE, "ERROR_MSG_FILE"
);
throw FileError("snf_EngineHandler::scanMessageFile() FileEmpty!");
}


Another clue is that in the log snippet you provide, there are hints
of a problem brewing when there are sporadic instances of this
error. Then, when there is a large block -- virtually all requests
to open the files for scan are rejected by the OS. Either something
made those files unavailable, or the OS was unable to handle the
request. I find it interesting also that the time required to report
the error started at about 172 milliseconds and continued to climb
to 406, 578, and then 656 before the restart.

SNF does not make log entries in the classic log during a restart,
by the way.

Note also the timestamps associated with these events and you can
see that the event was precipitated by a dramatic rise in message
rates. The first part of your log seems to indicate about 7-10
messages per second. During the large block of errors, the message
rate appears to have been in excess of 120 (I counted approximately
126 at timestamp 20130627183819). That's an increase at least an
order of magnitude higher than the rate that was causing sporadic
errors.

I suspect based on the data you have provided that something on your
system generated a very large spike of activity that your IO
subsystem was unable to manage and this caused snf scans to fail
because snf was unable to open the files it was asked to scan.

Your restart of SNF apparently coincided with the event, but since
all of the SMD file names are unique during the event, and since SNF
has no way to generate scan requests on it's own, SNF does not
appear to have been the cause of the event in any way. It was able
to record the event, none the less.

So the question in my mind now is: 

* Is there a way to improve your IO subsystem so that it can gain
some headroom above 10 msg/sec?
* What caused the sudden dramatic spike that led to this episode?

On most tiny systems I monitor, scan times are generally  100
ms. On your system they are frequently in excess of 400 ms which
leads me to believe your system is a bit underpowered for it's
current load.

Hope this helps,
_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message

[sniffer] Re: Slow processing times, errors

2013-06-28 Thread Pete McNeil

On 2013-06-28 16:49, Matt wrote:

maybe the updates will cause a service restart/reset?


Rulebase updates (all updates in fact) happen without restarting 
anything. SNF simply loads the new configuration, validates it, uses it 
for new scans, and when all of the old scans are complete it drops the 
old data. All of this happens without impacting scan operations.


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Slow processing times, errors

2013-06-27 Thread Pete McNeil

On 2013-06-27 17:25, Darin Cox wrote:
When we had sluggish performance similar that yours, resulting in 
numerous sniffer .tmp files in the spool, the cause was eventually 
traced to a proliferation of files in the sniffer directory.  Clearing 
them out brought performance back up to normal.


This is usually the problem. NTFS performs very badly when there are a 
lot of files in a directory -- and that slows everything down.


If SNF takes 30 seconds or more to process a message then SNFClient will 
give up and let the message through (fail safe).


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Slow processing times, errors

2013-06-27 Thread Pete McNeil

On 2013-06-27 16:04, Matt wrote:

like this:

20130627155608, arg1=F:\\proc\work\D6063018a2550.smd : Could 
Not Connect!


That is SNFClient giving up after waiting for SNF to process the message 
for too long.




At the same time, my Sniffer logs start showing frequent 
ERROR_MSG_FILE results on about 1/8th of the messages. 


This is SNFServer giving up after trying to open the message file and 
read it.


What's happening is that the OS is not delivering the file to SNF, SNF 
is waiting for this (it has no choice, it's a call to the OS's open() 
command), and then eventually it fails so SNF produces the 
ERROR_MSG_FILE result because it was not able to open the file it was 
supposed to scan.


This is often caused by fragmentation, or it can be that there are too 
many files in the directory that contains the message file. Ultimately 
it is an IO problem.


This shouldn't be associated with updates -- but if it is, I might guess 
that's because the file system is ready to fall over and saving a new 
rulebase file to disk, or reading afterward is enough to push it over 
the edge.


Seeing ERROR_MSG_FILE on 1/8th of the scans means that SNF is being 
asked to scan a message that the file system can't or won't give it. 
That is a strong indication that the system is IO bound. SNF can't 
really do anything different in that case -- it's simply asking to open 
the file so it can read it. If the IO system says No then it spits out 
that error.


Hope this helps,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: 2nd level IP scanning

2013-06-07 Thread Pete McNeil

  
  
On 2013-06-07 18:16, Peer-to-Peer
  (Spam-Filter.com) wrote:


  Hey Pete and all,
  
  Is there an option to have SNF scan second or
third deep header IPs? Im trying to block an originating IP
(66.83.88.42), however they are hopping thru Comcast and
Verizon.


Yes! You can use drilldown directives to teach SNF to "trust"
intermediate servers and find the originator:

http://www.armresearch.com/support/articles/software/snfServer/config/node/gbudb/training/drilldown.jsp

_M
    
    -- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: 2nd level IP scanning

2013-06-07 Thread Pete McNeil

  
  
On 2013-06-07 19:18, Peer-to-Peer
  (Spam-Filter.com) wrote:


  Im

  seeing one spammer whos IP address is (x.x.x.x). Or at least
  thats the originating IP in the headers. Im seeing
  thousands of messages originating from this IP, however they
  are passing thru hundreds of different Verizon and Comcast
  servers. Literally.
  
  I
  cant block (or I dont want to block) the Verizon or Comcast
  IPs, but I need to block the originating IP (x.x.x.x).


I think you have misunderstood what drilldown does.

If you teach drilldown to recognize the versizon and comcast servers
then it will learn to ignore them and pinpoint this specific IP. It
will also learn to find any other IPs that are doing the same kind
of thing.

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: IP Change on rulebase delivery system

2013-05-24 Thread Pete McNeil

On 2013-05-24 08:38, Richard Stupek wrote:

Pete
I thought the local gbudb got updates from the service or was that a 
future enhancement?


That's true right now. GBUdb is part of a distributed machine learning 
system. There is a conversation going on between all SNF nodes where 
they share their point of view on IP reputations. This happens 
approximately once per minute, out of band.


Each node alerts the system that they have new activity on a given IP. 
Then, via the SYNC server(s), each node receives a reflection of the 
consensus on that IP. So, when an IP is new to a node it will be updated 
within about a minute with the consensus reputation from the other 
nodes. As there are more interactions, the consensus matters less and 
the local experiences matter more -- but the conversation continues so 
the each node is always influencing the other nodes about any active IPs.


The conversation protocols are intelligent so that there is just enough 
traffic to accomplish the learning goals and so that a hostile / 
compromised node cannot poison the system; and so that each node can 
maintain it's own point of view about each IP.


For example: Say node A regularly corresponds with an ISP in 
blackhatistan. So, node A sees a mixture of good and bad messages. Node 
B only gets bad messages from the same ISP. Node A will have a local 
reputation for the ISP that is good enough to let messages through on 
that system, but node B will have a local reputation for the ISP that 
blocks most messages. The consensus of all GBUdb nodes will be somewhere 
in between.


Hope this helps,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-05-23 Thread Pete McNeil

On 2013-05-23 15:22, Richard Stupek wrote:
Looks like I have this issue again (pegging 4 core cpu) and resetting 
the process doesn't make a difference.  Not sure what is causing it 
but it does slow down spam detection to 40-50 seconds for many emails. 
 Any ideas what I can look at or do to resolve this?


Check the message sizes. As part of the newest spam storms we've noticed 
that a lot of the messages are huge (65536++). I suspect this might 
impact throughput as large buffers are allocated and moved around to 
handle these messages. This kind of thing has also been known to cause 
NTFS to crawl.


Please let us know what you find.

If you are not already doing it -- you should consider blocking 
connections using the truncate blacklist. No sense taking on some of 
these messages if they can be eliminated up front.


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-05-23 Thread Pete McNeil

On 2013-05-23 16:41, Richard Stupek wrote:
Can you point me at the documentation for the truncate blacklist and 
its usage?

http://gbudb.com/truncate/index.jsp

It's an ordinary ip4 dnsbl.

Most email systems have some mechanism for blocking connections based on 
this kind of blacklist.


Hope this helps,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-05-23 Thread Pete McNeil

On 2013-05-23 17:21, Richard Stupek wrote:
Would this: 
http://armresearch.com/support/articles/software/snfServer/xci/gbudb.jsp yield 
the same results as using the ip4 blocklist?


No. Asking your local GBUdb about an IP will only give you a local 
perspective.


The truncate blacklist contains the currently active worst-of-the-worst 
as seen by all SNF nodes working together.


Also -- getting your MTA to pay attention to your local GBUdb is 
nontrivial since no MTA software (that I know of) can speak XCI yet.


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Rulebase Compiler Improvements

2013-04-29 Thread Pete McNeil

Hello Sniffer Folks,

We have improved our rulebase compiler scheduling and efficiency.
This has allowed us to increase the pace of rulebase updates by 
approximately 20%.
You should see a further reduction in leakage rates and slightly more 
frequent rulebase updates.




Also: Did you know that Message Sniffer is designed not only to filter 
spam, but also to filter any email containing malware or viruses?


We recently performed a spot check by throwing a corpus of 200K known 
virus emails at SNF. Only 76 leaked. According to that and other similar 
tests SNF typically captures 99.962% of infected email.


This means that for most systems using SNF there is no need to also scan 
for viruses on the mail server -- SNF does it all at once. Anything that 
does get by SNF should be captured by desktop virus protection -- 
something nobody should do without since email is not the only way 
viruses can get into your systems.


It's important to note that infected messages may not be marked as 
malware -- SNF is designed learn and look for all kinds of unusual 
artifacts that help to identify unwanted messages and many of those are 
used not only in malware but also in other kinds of spam. So, a lot of 
the time infected messages are captured by patterns that were learned 
while looking at ordinary spam.


Please let us know if there is more we can do.

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Volume

2013-04-26 Thread Pete McNeil

On 2013-04-26 13:12, Peer-to-Peer (Spam-Filter.com) wrote:

Anyone else seeing the same?


The spamtrap pre-filter volume is about 4x typical. It's really quite 
something.


The stock-push stuff has a lot to do with it -- since that's become 
popular again we've seen striking volumes associated with it.


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Upgrading Stand-Alone Sniffer (for Declude)

2013-04-18 Thread Pete McNeil

On 2013-04-18 02:52, Andy Schmidt wrote:

SNFMulti Engine Version 3.0.11 Build: Aug 21 2009 18:42:53
SNF Server Version 3.0.2 Build: Jul 28 2009 14:48:00


That is currently the latest official release.

There is a slightly newer SNFServer.exe that is an interim (snapshot) 
release:


http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.0.23.zip

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Reputation Lookup DNSBL?

2013-04-18 Thread Pete McNeil

On 2013-04-18 03:06, Andy Schmidt wrote:

Is there a GBUdb IP based lookup that is recommended to get the benefit of
all Sniffer customers' experiences?


There is the truncate blacklist http://www.gbudb.com/truncate/index.jsp

Other than that SNF will automatically learn what the other SNF nodes 
know about an IP within about a minute of the first encounter. Then as 
your SNF node has more experience with the IP it will begin to trust 
it's own data more than that of the other nodes.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IPScan results

2013-04-16 Thread Pete McNeil

On 2013-04-16 17:02, Peer-to-Peer (Spam-Filter.com) wrote:
Hey all, I noticed a couple of my MDaemon mailservers are not 
performing the “SNF IPScan”.


Check that your MDaemon versions are the same -- some versions implement 
the plugin API differently.


Then make sure that your Plugins.Dat file configures the SNF plugin 
correctly.


If you've got one server working correctly and other's not, then that 
gives you a good way to compare.


Hope this helps,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IPScan results

2013-04-16 Thread Pete McNeil

On 2013-04-16 20:40, Peer-to-Peer (Spam-Filter.com) wrote:

Once I installed a new Security Plus license, Outbreak Protection engine
started and so did SNF IPScan.


Around 20120905 I asked about this and Arvel put in a patch to remove 
the restriction.


That is, since Md 13.0, you do not need the security plus license to use 
the full API on plugins, so the original SNF4MDaemon plugin design will 
work. (that's what you have configured).


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Convert your Declude OEM license now and get full credit!

2013-04-11 Thread Pete McNeil

On 2013-04-11 08:22, e...@insight.rr.com wrote:

Because of this entire issue with declude. It might be nice if you contacted
smarterTools and offered to work with them on them integrating message
sniffer directly into smarterMail. :)

We have attempted this on several occasions and it hasn't worked out.

We remain optimistic and ready to work with them if they decide to 
change their minds about it. I know that SNF's features and speed, if 
fully integrated, would take SM spam and malware filtering to the next 
level.


In the mean time, it is possible to integrate SNF directly with Smarter 
Mail by calling it as a command line scanner. Then the injected headers 
can be used in filtering rules or to add weight to the built-in 
SpamAssassin scores.


http://www.armresearch.com/support/qa/integration/smarterMail.jsp

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Convert your Declude OEM license now and get full credit!

2013-04-10 Thread Pete McNeil

Hi Sniffer Folks,

It appears that Declude (the company) is failing. After many rumors of 
problems and some first hand experience, today the Declude web site has 
gone dark.


We have a long standing relationship with the Declude community, and we 
want to make sure we do what we can to support them even if Declude 
itself goes away.


Place a new order for Message Sniffer (SNF) now and we will give you 
credit for any time you have left on your Declude OEM license. Tell us 
your OEM expiration date with Declude and we will add the time you have 
left to your new SNF license.


For the best pricing we recommend you purchase through one of our resellers:
https://www.armresearch.com/products/resellers.jsp

Please be sure to pass this information on to any interested folks that 
might not be on this list! There is bound to be a lot of turmoil right 
now and we don't want anybody to miss it.


Please let us know if there is more we can do!

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-03-29 Thread Pete McNeil

On 2013-03-29 12:59, Richard Stupek wrote:
well when all else fails restarting snf seems to have corrected the 
issue for now.


In that case, it is likely that RAM fragmentation was involved. Dropping 
the process allowed the fragmentation to be cleared. (theory).


Best,
_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-03-28 Thread Pete McNeil

On 2013-03-28 12:10, Richard Stupek wrote:
Ok looking at the log I see quite a few messages taking over a second 
to process (samples below):


s u='20130328155503' m=\temp\1332407477322.msg' s='0' r='0'
p s='1172' t='1109' l='72697' d='127'/
g o='0' i='12.130.136.172' t='u' c='0.486243' p='-0.625' r='Normal'/
/s


Great! Now we're getting somewhere.
It seems likely that your system is bound in 2 ways:

* Allocating RAM
* Reading / writing from the hard drive.

The setup time s='1172' indicates that it took more than a second to 
allocate a 72K buffer and read the message. That is the only work done 
during setup.


The scan time t='1109' indicates the amount of time it took to complete 
the rest of the process. I'm guessing that since the setup took more 
than a second that writing the message back with injected headers 
probably took a while too.


A scan depth of 127 is nominal.

The data you sent indicates this is a problem when the messages are 
fairly large. There is likely a boundary condition between smaller 
messages and larger ones that allow the smaller messages to be handled 
more efficiently.


Since you are indicating that CPU utilization is high during these 
events and since you're not mentioning other performance issues, it 
seems likely that RAM fragmentation or RAM starvation might be the 
problem here.


During the setup, SNF allocates a buffer large enough for the entire 
message and then reads it all at once. This is most efficient because 
there is a single system call for each of these events - so the OS has 
complete control over the entire step and it only happens once.


After the scanning process is done, SNF will typically allocate another 
buffer large enough to include the message and all of it's headers. This 
new version of the message is constructed in the buffer and then written 
all at once to the file system.


If the problem is RAM fragmentation or starvation then it could be 
taking as much as a second for the OS to allocate a 72K buffer --- 
that's the kind of thing that should be nearly undetectable, but in 
these cases where high CPU loads are reported with this kind of log data 
I have seen that happen.


If the problem were simply IO then it is likely that CPU utilization 
would be low while the CPUs wait for the IO operations to happen.


Of course it could be a combination of things -- it's hard to tell 
what's happening in the internals of the OS.


Hope this helps,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-03-27 Thread Pete McNeil

On 2013-03-27 14:38, Darin Cox wrote:
Probably unrelated... and due to a significant increase in spam over 
the past few days.


I agree with that -- our inbound spamtrap pre-processor has seen 4x 
increase over the past few days so that's likely to be related.


Also, Richard, I took a quick look at your telemetry and verified that 
your rulebase file(s) are up to date.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-03-27 Thread Pete McNeil

On 2013-03-27 16:49, Richard Stupek wrote:
Its odd because the number of messags snf is processing isn't more 
than usual and the % of spam being detected through snf is actually 
lower than typical yet is is routinely maxing out 4 processors at 100%.


You're saying that SNF is maxing out 4 processors? ... or is the 
combination of operations on your server maxing out 4 processors?


We're using the same engine and ruelbase in our CGP server and humming 
along nicely at between 2000 - 8000 msg/minute with nominal CPU loads.


I don't see anything unusual in your telemetry and I haven't heard any 
other complaints, so I can't explain why SNF would act differently on 
your system. I hate a mystery though -- so I would love to get to the 
bottom of it.


Do you see anything else that might be causing the CPU load?

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: IP Change on rulebase delivery system

2013-03-27 Thread Pete McNeil

On 2013-03-27 17:16, Richard Stupek wrote:

The spikes aren't as prolonged at the present.


Interesting. A short spike like that might be expected if the message 
was longer than usual, but on average SNF should be very light-weight.


One thing you can check is the performance data in your logs. That will 
show how much time in cpu milleseconds it is taking for each scan and 
how long the scans are in bytes. This might shed some light.


http://www.armresearch.com/support/articles/software/snfServer/logFiles/activityLogs.jsp

Look for something like p s='10' t='8' l='3294' d='84'/ in each scan.

From the documentation:


sp//s - Scan Performance Monitoring (performance='yes')
p:s = Setup time in milliseconds
p:t = Scan time in milliseconds
p:l = Scan length in bytes
p:d = Scan depth (peak evaluator count)



Best,

_M


--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] IP Change on rulebase delivery system

2013-03-25 Thread Pete McNeil

Hi Sniffer Folks,

We are about to change the IP of the rulebase delivery system. This 
change should be completely transparent and you should not need to take 
any action; however if you do notice anything unusual please let us know.


Thanks,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] GBUdb Tool

2012-11-23 Thread Pete McNeil

Hello Sniffer Folks,

We have been playing with a new utility that some of you may enjoy.

http://www.armresearch.com/message-sniffer/download/GBUDBTool-V0.1.zip

GBUDB Tool allows you to create a list of IP addresses from your GBUdb 
snapshots (.gbx files). You can select IPs that are blacker or 
whiter than a provided probability figure and confidence figure. It 
outputs one IP per line, optionally with details about the statistics 
for the IP. This can be useful for feeding-forward blacklists to block 
at your firewall or for other research purposes.


Run GBUDBTool without any parameters and it will tell you about it's 
command line options.


Please let us know if there is more we can do.

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] SNFServer Interim Release E3.0.23

2012-11-23 Thread Pete McNeil

Hello Sniffer Folks,

Here is a link to an interim release of SNFServer for Win* boxen:
http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.0.23.zip

This interim release fixes a bug in the previous E3.0.19 interim where 
large messages might be corrupted during message header injection. This 
new version has been tested thoroughly against large messages.


If you don't recall, the E3.0.19 interim and above allows for up to 8 
messages to be scanned simultaneously when sufficient CPU cores are 
available.


If you are running *nix and would like to try the interim version then 
feel free to pull down the updated SNFMulti source code from the SVN server:


https://svn.microneil.com/websvn/filedetails.php?repname=SNFMultipath=%2Ftrunk%2FSNFMulti.cpp

It is not necessary to upgrade your SNF installation if you are not 
running one of the interim releases. If you are running a production 
release then you're good to go as you are.


Please let us know if there is more we can do.

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] High Throughput Windows version of SNFServer available

2012-09-28 Thread Pete McNeil

Hi Sniffer Folks,

If you have a high volume Windows based mail server and you would like 
to beta test some performance features then you can find a new version 
SNFServer here:


http://www.armresearch.com/message-sniffer/download/SNFServerV3.0.2-E3.0.19.zip

There are two key features we're testing on this version:

* If header injection is turned off then the message file will only be 
read up to a maximum of 32K (or the current scan horizon).


* There are now 8 processing channels in the XCI server so that 8 
simultaneous scans can occur if sufficient cores are available.


If you decide to test this then please let us know.

Thanks!

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Creeping higher on those rule numbers

2012-06-26 Thread Pete McNeil

On 6/26/2012 9:41 PM, Colbeck, Andrew wrote:

Rule number 5 million rolled on by this week.

Yes indeed!
_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Creeping higher on those rule numbers

2012-06-26 Thread Pete McNeil

On 6/26/2012 9:41 PM, Colbeck, Andrew wrote:

Rule number 5 million rolled on by this week.
Message Sniffer Rule # 500 was coded by Andy (Worm Thunder) 
20120626.1408 SortMonsters Rock!


I wonder who won the pool?

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: FPs on Sniffer-Schemes

2012-03-13 Thread Pete McNeil

  
  
On 3/13/2012 11:19 AM, Scott Fosseen [Prairie Lakes AEA] wrote:
Can you check to see if all looks ok with
my copy as well.
Sure. I'll respond off-list
_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: FPs on Sniffer-Schemes

2012-03-12 Thread Pete McNeil

  
  
On 3/12/2012 5:17 PM, Darin Cox wrote:

  
  
  
  Hi Pete,
  
  We're seeing a ton of FPs on a
  Sniffer-Schemes rule # 4764784.


That rule was detected as an error and removed almost immediately
after it was created.
You should not be seeing any additional hits on that rule.

Best,

_M


-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: FPs on Sniffer-Schemes

2012-03-12 Thread Pete McNeil

  
  
On 3/12/2012 5:41 PM, Darin Cox wrote:
Started getting hits at 4:30pm EST up to
15 minutes ago (5:25pm EST). Not sure if the rule has been
pulled or corrected yet.
It was corrected nearly as soon as it was created. It did escape
into some rulebases - we saw that on our conflict instrument. Most
systems auto-panicked the rule right away. It no longer appears on
our conflict instruments - so there is no reason you should see any
hits from it.

I'm chasing things down to see what I can see -- based on your
message.

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: FPs on Sniffer-Schemes

2012-03-12 Thread Pete McNeil

  
  
On 3/12/2012 5:41 PM, Darin Cox wrote:
Started getting hits at 4:30pm EST up to
15 minutes ago (5:25pm EST).
I think I can see part of the problem (possibly).
I do not have telemetry from your system (based on looking up your
Id from your domain). I suspect this means that you are running an
older version of SNF. By extension, that would mean a couple of
things:

* Your rulebase update would not come as quickly as for most
systems.
* Your SNF engine won't match on many of the newer rules.
* Your SNF engine will not have GBUdb and also will not be able to
auto-panic new rules that conflict with IP reputation data.

Am I right about these assumptions?
If not, then we should figure out why I don't see your telemetry.

Thanks,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 x7010
twitter/codedweller 

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Bad rule event

2012-02-22 Thread Pete McNeil

Hi Sniffer Folks,

We had a short bad rule event this morning.
The following rule IDs were matching unintended text, they were 
discovered quickly on our conflict instrumentation and removed after 
approximately 30 minutes. Most systems were rejecting the rules (that's 
how the conflict instrument gets it's data).


4711347
4711362
4711345
4711346
4711360
4711361

We have identified how the rules were coded and adjusted our practices 
to make this less likely in future. Also, our system will remember these 
rules automatically so that we cannot make the same mistake again.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] System Upgrades

2012-02-21 Thread Pete McNeil

Hello Sniffer Folks,

Just a quick note to let you know:

* We have boosted our rulebase production system. New rulebase updates 
will arrive about 25% faster on average.


* We have optimize Rulebot productivity to respond to a wider range of 
spam / malware variants automatically.


* We have augmented our QC processes to seek out more potential false 
positive cases and stop them before they occur.


* We have added additional research channels to help identify more 
threats more quickly.


---

Note that over the next few weeks we will be making additional changes 
to our technical infrastructure. During service windows occurring at 
times of low-activity there may be short disruptions in SYNC server 
connections and/or rulebase delivery. We will do our best to avoid 
these, and those that do occur should go unnoticed.


Your Message Sniffer software installation is designed for high 
performance and high availability. It will continue to function normally 
even if we have a disruption during our upgrades, and it will 
automatically recover from any such disruption without any assistance.


Please let us know if there is more we can do.

Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Ok, I'm the 3rd person to ever report the Bad Matrix error on this mailing list

2012-01-09 Thread Pete McNeil

  
  
On 1/9/2012 4:15 PM, Colbeck, Andrew wrote:

  
  
  


snip/


  
If there's here for the
  SortMonsters, it's to make sure that a "bad matrix"
  error doesn't interfere with downloading a fresh
  rulebase so that SNFserver.exe can get itself out of
  that jam.
  


SNFServer will reject a bad rulebase and keep running with the old
one. So, if somehow a bad rulebase shows up then SNFServer won't
crash... it will keep trying to get a new rulebase with the
getRulebase script until it is successful. By default it will try
about once every 3 minutes if it's not initially successful.

However, once SNFServer is down for any reason, it will refuse to
start without a good rulebase file to work with.

The SNFClient utility can only attempt to ask SNFServer to scan a
message -- it can't do it by itself. If SNFClient is not able to
contact SNFServer and get a good answer then it will create
SNFClient.exe.err to explain the problem and will return 0 to the
calling program -- It returns 0 as a fail-safe so that the messages
will go through. Better to allow all messages through than to block
any good messages by mistake.

On a well functioning system there should [almost] never be an
SNFClient.exe.err file. I say [almost] because things happen from
time to time on every system.

However, if you see a .err message, check it out. If they persist -
something is wrong.

If you try to start SNFServer and it is unhappy, then download a
fresh rulebase first. It's usually a good quick-fix.

Best,

_M
    -- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 
x7010
  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: Training GBUdb on the client IP for telus.net

2011-10-24 Thread Pete McNeil

On 10/24/2011 2:46 PM, Colbeck, Andrew wrote:

would this snippet in snf_engine.xml

I don't see the snippet from snf_engine.xml?
_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Training GBUdb on the client IP for aol.com

2011-10-24 Thread Pete McNeil

On 10/24/2011 3:21 PM, Colbeck, Andrew wrote:

header name='X-Originating-IP:' received='.aol.com [' ordinal='0' /

As far as I know that one still works.
_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Training GBUdb on the client IP for telus.net

2011-10-24 Thread Pete McNeil

On 10/24/2011 3:20 PM, Colbeck, Andrew wrote:

header name='X-Telus-Outbound-IP:' received='.telus.net [' ordinal='0'
/

Which is in the GBUDB/Training/Source section as per:

http://www.armresearch.com/support/articles/software/snfServer/config/no
de/gbudb/training/source-header.jsp

That appears to be correct and appears to have worked correctly.
Top Received header would have been picked as source IP (unless you 
already have it ignored).
It appears that you have successfully told SNF to find the source IP in 
the X-Telus-Outbound-IP: header in this case.


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Training GBUdb on the client IP for telus.net

2011-10-24 Thread Pete McNeil

On 10/24/2011 3:20 PM, Colbeck, Andrew wrote:

header name='X-Telus-Outbound-IP:

Hrmm... Do you want the source to be the outbound IP?

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] SNF Server / Client for *NIX updated - IMPORTANT bug fix included

2011-09-26 Thread Pete McNeil

Hello Sniffer Folks,

Tarball snf-server-3.0.13.tar.gz has been posted to the Products Page:

http://www.armresearch.com/products/index.jsp

Or you can download it directly here:

http://www.armresearch.com/message-sniffer/download/snf-server-3.0.13.tar.gz

This distribution contains some minor bug fixes and code improvements 
bringing the SNFMulti Engine up to 3.0.17.


IMPORTANT: This distribution also contains a clean SNFServer/main.cpp 
that fixes a random crash bug!


The previous distribution snf-server-3.0.12 contained testing code that 
would intentionally force a crash (seg fault) under some specific load 
conditions. The testing code would make it appear that SNFServer was 
crashing at random with crashes being more likely under higher load 
conditions.


This testing code should not have escaped the lab and was not intended 
for use in production. We have reviewed and revised our publishing 
procedures to ensure this does not happen again. This new distribution 
snf-server-3.0.13 does not contain the testing code.


This bug was not included in Win* distributions - only 
snf-server-3.0.11.tar.gz and snf-server-3.0.12.tar.gz included the 
errant testing code.


Thanks!

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Bug Report: SNFServer for *nix

2011-09-22 Thread Pete McNeil

Hello Sniffer folks,

We have discovered that some testing code escaped into the latest 
tarball: snf-server-3.0.12.tar.gz


This testing code intentionally causes SNFServer to crash (seg fault) 
under special conditions. This was done so that we could examine the 
resulting core dump.


You may experience this as random crashes, especially during spikes in 
traffic.


We are looking at our procedures to see how this happened. When we have 
resolve that issue we will publish a new tarball.


In the mean time you can correct this problem immediately by replacing 
your SNFServer/main.cpp file with the one found on our SVN server here:


https://svn.microneil.com/websvn/dl.php?repname=SNFServerpath=%2Ftrunk%2FSNFServer%2Fmain.cpprev=16peg=16;

Download the file, rebuild SNFServer, and replace your existing 
SNFServer.exe.


Sorry for the trouble.

We will get a new tarball out shortly.

Thanks!

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Bad Matrix errors

2011-08-22 Thread Pete McNeil

  
  
On 8/22/2011 4:04 PM, Peer-to-Peer (Support) wrote:

  
  
  
  HelloSNF,
  
  I think something broke.I'm
  seeing a lot of "Bad Matrix!" warnings in my logs.Likely startedabout an
  hour ago.
  Running MDaemon
  mailserver.



I note in your telemetry that you have a new rulebase since then.
Have the errors stopped?

_M

    
    -- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 
x7010
  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Change in default settings

2011-05-09 Thread Pete McNeil

Hello Message Sniffer Folks,

We're recommending a change in the default settings for message sniffer 
in order to improve our response times for new campaigns. The change is 
small and enhances our virtual spamtrap technology so that we see new 
spams sooner and with greater sampling coverage.


If you locate this block of code in your snf_engine.xml file:

black on-off='on' symbol='63'
edge probability='0.8' confidence='0.2'/
edge probability='0.8' confidence='1.0'/
truncate on-off='on' probability='0.9' peek-one-in='3' symbol='20'/
sample on-off='on' probability='0.8' grab-one-in='3' passthrough='no' 
passthrough-symbol='0'/

/black

You will notice that your settings are probably slightly different.

The changes we would like you to make are:

peek-one-in='3'
grab-one-in='3'

Your current settings most likely use higher numbers for these settings.

Once you make the change and save your file then Message Sniffer should 
pick up the changes right away - you do not need to restart Message 
Sniffer when making adjustments to your configuration.


Please let us know if you have any questions.

Thanks!

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Change in default settings

2011-05-09 Thread Pete McNeil

On 5/9/2011 3:43 PM, Peer-to-Peer (Support) wrote:

Hi Pete,

Just double checking:  My snf_engine.xml file does not have any 'single
quotes' around any numbers or characters.
See attached as an example.


What you have there in that png is your configuration log -- it is SNF's 
interpretation of your configuration file. The actual configuration file 
does use single quotes (unless you changed it).


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Change in default settings

2011-05-09 Thread Pete McNeil

On 5/9/2011 4:53 PM, Colbeck, Andrew wrote:

Pete, for

sample on-off='on'

I wrote myself this note...

!-- We can sample during a peek if passthrough = yes --

... Is it still valid? Your sample and my own configuration have:

passthrough=no

On the balance of it, I suspect my own note is wrong, so it would be
nice if you could verify it one way or the other.


The passthrough option is for local sampling. We have used it 
occasionally on our spamtrap processors, but not for some time. 
Passthrough takes any messages that would have been samples and instead 
of sending them to the virtual spamtrap network it lets them go through 
with a specific result code. Presumably the local system would see the 
special result code and treat the message differently.


Please leave passthrough='no'

Thanks!

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] IMail mail1.exe removed

2011-04-13 Thread Pete McNeil

Hi Sniffer Folks,

Many of you who use Imail also use the imail1.exe command line utility 
to send yourself updates and reports from your system. Be advised (if 
you don't already know) that this program has been removed from IMail 
and there does not appear to be a replacement at this time.


Here is a discussion:

http://forums.ipswitch.com/Topic58862-10-1.aspx#bm58935

One customer who recently upgraded IMail discovered this problem when 
their update reports stopped working.


The work-around was as follows:

1. Locate a backup of your previous IMail installation.
2. Locate the imail1.exe and imailsec.dll files from your backup.
3. Create a sub-directory inside your SNF directory (suggest IMAIL1).
4. Copy the imail1.exe and imailsec.dll files into the new 
sub-directory. DO NOT copy them into your IMail directory.
5. Modify your scripts to call imail1.exe in the new location using the 
full path, or if the script runs inside of your snf directory you can 
use a relative path (like imail1/imail1.exe).


This should restore the original functionality.

There are also command line utilities around for sending messages, but 
for long-time IMail users this is probably the easiest solution.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: So, another botnet bites the dust.

2011-03-18 Thread Pete McNeil


  
  
On 3/18/2011 4:17 PM, Colbeck, Andrew wrote:

  
  
  Pete,
now that Microsoft has taken down the Rustock botnet, what's
your telemetry say about spam volumes? Any significant
change?


I don't see a dent.





-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 
x7010
  



[sniffer] Re: IPv6

2011-03-11 Thread Pete McNeil

On 3/11/2011 9:44 AM, Bonno Bloksma wrote:

Hi,

I remember reading somewhere research was being done about ipv6 block lists
using the fact that the same /64 net would probably be the same machine or
very near it. Prety much what we now Block when we list an ipv4 NATted
gateway to a private network which houses an infected PC.



I had some heated conversations about this at HostingCon in Texas last 
year. The fellow was pushing IPv6 preparedness and had not considered 
some of the issues I raised about it.


As I see it the real trouble is going to show up where IPv6 and 
virtualization meet. Consider that to keep costs and manageability 
reasonable a /64 (or some largeish sub block) will be allocated to a 
chunk of hardware hosting virtualized machines. Consider also that 
virtualization is being highly optimized. Therefore an attacker (spammer 
etc) will be able to map themselves a virtually inexhaustible supply of 
IP addresses and hosts and move them around at will.


Even if they violate TOS on a legitimate system they will be able to do 
it for such a short period of time with such a small data footprint that 
they will be able to remain undetected for long periods-- perhaps even 
indefinitely. By the time anybody looks to see what is going on the 
offending VPS will long since have been destroyed and it's cousin will 
be living in a completely different data-center picking up where it left 
off.


The good news is that this behavior is still dramatically different than 
that of legitimate senders so it leaves a statistically important footprint.


What might be predicted from this? Off the top of my head I think maybe 
the following...


* Attacks from anonymously controlled virtual bots will rise 
dramatically and will be very difficult to defend. Consider that we 
currently have sufficient RAM available to completely bitmap every IP4 
address if we choose to do so -- and that could be done at wire speed on 
even the fastest routers (Still can't get any love for the concept, but 
it's been possible for a long while now). This will not be possible with 
large scale IPv6 deployment.


* Black-listing will become softer and much more difficult. Due to the 
convergence of virtualization and IPv6 deployment, IPs of legitimate 
systems will necessarily merge with the IPs of illegitimate systems. 
Only specific, long-lived IPs from legitimate systems will be worth 
tracking.


* Legitimate systems that do bulk mailing will make increasing use of 
virtualization to keep costs down -- standing up large bot-nets of their 
own to deliver a campaign and then evaporating those bot-nets when 
delivery is complete. This will make such systems virtually 
indistinguishable from illegitimate senders since the IP blocks will 
have significant overlap and the usage statistics will be very similar.


* White-listing mechanisms will become more important.

* Content analysis will become more important. SNF is good at that :-)

* Systems that delay delivery of messages from unknown and untrusted 
systems will be more important -- especially those that allow for 
delivery after re-scanning content rather than conventional 
gray-listing. Conventional gray-listing mechanisms will become more 
difficult to use because all kinds of legitimate bulk mailing systems 
simply will not be there to re-send undelivered messages due to the 
systems being shut-down after the first volley in order to contain 
costs. I expect some significant increases in the complexity of such 
systems to compensate for this.


* On the way to IPv6 there will be a lot of fragmentation and confusion 
of all types. For a long time to come some folks will be unable to 
deploy IPv6. Others will be unavoidably required to do so. Bridges 
between these networks will be necessary, difficult to regulate, and 
unpredictable. Best practices for IPv6 and mixed networks will be 
difficult to define and constantly evolving -- this churn in general 
will slow adoption and increase fragmentation.


What other things might we expect?

Anybody think one or more of these predictions are unrealistic? If so, 
why? After all, it's just conjecture at this point ;-)


In any case, SNF is evolving to become ever more intelligent and 
adaptive. We will concentrate not only on more sophisticated content 
analysis, but also behavioral analysis and an increasingly cognitive 
approach to blending data from all of these subsystems and responding in 
realtime.


_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig

[sniffer] CommuniGate Pro Plugin for MS Windows Updated

2011-01-17 Thread Pete McNeil

Hi Sniffer Folks,

Just a quick note to let you know we've updated the MS Windows version 
of our Anti Spam / Anti Malware plugin for CommunigGate Pro.


http://www.armresearch.com/products/SNF4CGP.jsp

http://www.armresearch.com/message-sniffer/download/CGPSNF-Win32-Intel.zip

We have rewritten the documentation and distribution files to make the 
installation process simpler and clearer.


We've also updated the main configuration file with CSS and XSL so that 
you can view a clear, human friendly version of your snf_engine.xml file 
simply by opening it in your web browser.


Enjoy!

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: RulePanic on 3741490

2011-01-07 Thread Pete McNeil


  
  
On 1/7/2011 10:19 AM, Darin Cox wrote:

  
  
  
  Hi guys,
  
  We're seeing a lot of FPs on
  3741490 this morning. I've added a RulePanic for it in our
  systems.


The rule was created at 0539 and removed at 0853 when it was
detected by our early warning system.
It codes for a binary segment found in some image files.

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 
x7010
  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: RulePanic on 3741490

2011-01-07 Thread Pete McNeil


  
  
On 1/7/2011 12:33 PM, Darin Cox wrote:

  
  
  Hmmm... so 70 minutes after the
  rule was released we were notified of the rule update for
  auto-update of rulebase, but at 10:11ET we still hadn't gotten
  the update for the 8:53am removal. Anything we can do to
  speed up the rulebase update notifications?


Update notifications happen as soon as the rulebase compilers have
created a new rulebase. We are in the process of reworking our
compiler cluster to improve it's performance and further shorten
update times.


  
  Also, for rules identified as
  problematic and removed, what about an automated email so we
  can remove it immediately via RulePanic. For peak times like
  beginning of the business day, that would be very helpful. An
  hour could save a lot of headaches for both us and our
  customers. Or are there so many of those that we would be
  swamped with notifications?


We have features on the short list to automatically render removed
rules inert in near real-time (within seconds).


  
  Just trying to figure out a way
  to avoid this as much as possible in the future. It cost me a
  half hour this morning, and, more importantly, delayed over
  150 legitimate messages to our customers.


We are constantly improving our process to minimize these cases,
increase the speed with which we can detect and correct these, and
add features to automate and expedite the process.


  
  Thanks in advance for anything
  you can do.


Thanks very much for your feedback!

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 
x7010
  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Re: RulePanic on 3741490

2011-01-07 Thread Pete McNeil


  
  
On 1/7/2011 2:00 PM, Darin Cox wrote:

  
  
  
  H
  
  "Update notifications happen as soon as the rulebase
  compilers have created a new rulebase."
  
  I don't know what your internal
  processes are, butif I understand this correctlythe rule was
  created at 5:39am ET, and was compiled into the rulebase
  somewhere just before 8:53am ET, at which point update
  notifications were sent.
  
  From the customer point of view,
  when the rule was created or removed doesn't really
  matter,and those times are meaningless to us. What matters
  is when the rulebases that include them are published/updated,
  as that is what we key off of for updates.


The rulebase compiler that was responsible for your update would
have performed it's queries to collect rules for folding. For some
time after that the compilers would crunch on that data in order to
create the folded token matrix. Both the query and folding
operations take time. There are many compilers and many, many
rulebase files. The system is adaptive so the times required to
perform these operations changes constantly.

All that by way of saying that you are essentially correct, but also
that there is no single answer we can give that would describe the
case for every customer.

The rule was removed at about 0853. Some customers rulebase
compilation began immediately after that event, others before. All
took some time to complete.

Part of our rework is to reduce the time required for both phases of
this operation so that on average less time is required for a change
to appear in active SNF systems. The goal is ultimately to make that
as close to instantaneous as possible without incurring performance,
availability, scalability, and reliability penalties.


  
  "We have features on the short list to automatically
render removed rules inert in near real-time (within
seconds)"
  
  Sounds good. That would
  definitely be better than notifications for us to be able to
  put in RulePanics, assuming there's no negative effect to
  overall performance from checking each rule for
  active/inactive state. I assume some sort of push mechanism
  to all subscribers, to notify their systems that a rule is no
  longer valid,is what you're planning here.


The implementation that is currently planned would operate something
like our system making temporary entries into your rule-panic
system. Those entries would survive long enough to ensure that your
active rulebase contains those changes before the entries expire.

Wherever possible we like to leverage tried and true mechanisms
(like the rule-panic entry system) when producing new functionality.
Also, wherever possible we like to engineer facilities that can be
leveraged in multiple ways in future. It's a planning heavy process,
but one that pays off in better reliability and greater overall
flexibility. (IMO).

Best,

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 
x7010
  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Bad Rule Event

2010-12-16 Thread Pete McNeil

Hello Sniffer Folks,

We have had a bad rule event.
The bad rules were created near 0830E, and removed by 1030E.
The bad rules were discovered by our IP/Rule conflict instrument 
indicating that most were automatically rejected by Auto-Panic features.
The rules were part of a rule family designed to capture highly 
obfuscated porn subjects.
Unfortunately the extensive abstraction of the rules matched other 
subjects containing similar combinations of characters.

The Rule IDS are in the range 3694383 - 3694428.
We are very sorry for any trouble and have already taken measures to 
prevent this error in future.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Bad Rule Event

2010-12-16 Thread Pete McNeil


  
  
On 12/16/2010 11:07 AM, Bonno Bloksma wrote:

  
  
  
  Hi Pete,
  
   Hello Sniffer Folks,
   
   We have had a bad rule event.
   The bad rules were created near 0830E, and removed by
  1030E.
  [...]
  
  Regarding this event A while
  ago we talked about sniffer installations exchanging
  rule-panic info via the GUBdb sync info as that is happening
  every (few) minute(s) in stead of every few hours.
  Any idea when a new version of
  Sniffer with that feature will be launched?


Actually -- rule-panics are triggered instantaneously based on local
GBUdb data.

Auto-Panic:
When a relatively new rule conflicts with a known good IP on your
system that rule is made inert until the next rulebase update.



The next full release will include features for near-real-time rule
additions and removals.

We plan to begin releasing interim updates of the SNF engine with
some of these features early next year.

We plan to complete the next full release by Q3.

_M

-- 
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044 
x7010
  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




[sniffer] Hello again

2010-11-09 Thread Pete McNeil

Hello Sniffer Folks,

This is just a quick note to touch base. It's been a while since we've 
had an update, and the list is so quiet!


* Over the past few days we've finished a major re-tuning of our 
rulebase compiler system. The improved rulebase compiler bots are just a 
bit smarter and as a result many systems are receiving their updated 
rulebase files sooner than ever before. This means capturing more spam 
early on more systems and as a result more accurate data in GBUdb for 
new bot-nets. A win for everyone.


* We're growing again: Please welcome our newest SortMonster: Brian (The 
Fall Guy). We look forward to Brian's help as we expand our research 
capacity and improve our flexibility so that we can rapidly respond to 
new spam storms and preemptively defeat those we can predict.


* If you don't already know, a few months ago we launched 
truncate.gbudb. Truncate is an rfc5782 compliant DNSBL that contains 
IPs from the dark edge of Message Sniffer's GBUdb system. So far the 
comments we've received have been very positive. Be sure to give it a 
look when you get a chance and see if it can help you safely block 
connections at the edge of your systems. 
http://www.gbudb.com/truncate/index.jsp


* Over the next few months we will be upgrading our servers to increase 
our capacity and support more sophisticated analysis and research 
engines. All of this translates to improved rulebase quality and 
response times. Expect this round of upgrades to be finished in the 
first half of the new year when we will have some exciting surprises.


* Quietly in the back of the lab we have been building a sophisticated 
proactive false positive prevention system and we've been seeing some 
impressive results. Working closely with several of our largest partners 
we have developed a system which allows us to identify and eliminate 
many false positives before they can occur! This has allowed us to take 
our already low false positive rate and reduce it by nearly 30% over the 
past few months - especially focusing on low-level false positives that 
might otherwise never be reported or detected. As a result we have seen 
a measurable reduction in false positive related support traffic across 
all of the systems we monitor. We expect further improvements as we 
develop more sophisticated tools to help us with our research.


=

We would love to hear from you!

Please send us a note from time to time and let us know what we're doing 
right and how we can improve.


Best,

_M

--
Pete McNeil
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com
866-770-1044
x7010


#
This message is sent to you because you are subscribed to
 the mailing list sniffer@sortmonster.com.
This list is for discussing Message Sniffer,
Anti-spam, Anti-Malware, and related email topics.
For More information see http://www.armresearch.com
To unsubscribe, E-mail to: sniffer-...@sortmonster.com
To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com
To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com
Send administrative queries to  sniffer-requ...@sortmonster.com



[sniffer] Re: Testing SM direct intergration

2010-09-22 Thread Pete McNeil


  
  
On 9/22/2010 6:58 PM, Keith Dovale wrote:

  
  
  
  
Hi Guys, I would be interested in testing
  this ..
  

Here is a reminder of the link.

http://www.armresearch.com/support/qa/integration/smarterMail.jsp

Currently this is the only way to integrate SNF directly with
SmarterMail.

There are a number of 3rd party products which also allow SNF to
integrate with SmarterMail while providing additional functionality.

Perhaps in the future there will be a tighter integration with
SmarterMail that leverages SNF during the SMTP conversation in order
to take maximum advantage of what SNF can provide-- we hope so, but
it's not happening yet. If this sounds like a good idea to you be
sure to let the folks at SmarterMail know you think so.

Best,

_M
-- 
Chief Scientist
ARM Research Labs, LLC
www.armresearch.com

  

#

This message is sent to you because you are subscribed to

  the mailing list sniffer@sortmonster.com.

This list is for discussing Message Sniffer,

Anti-spam, Anti-Malware, and related email topics.

For More information see http://www.armresearch.com

To unsubscribe, E-mail to: sniffer-...@sortmonster.com

To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com

To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com

Send administrative queries to  sniffer-requ...@sortmonster.com




  1   2   3   4   5   6   7   8   9   10   >