[sniffer] Re: Beta
You nailed it Pete, Thanks! Darrell -- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Pete McNeil wrote: Hello Darrell, Tuesday, October 16, 2007, 9:26:47 PM, you wrote: Pete, Can you cover how the communication for the GBUdb system works? Sure. (And documentation is coming along with a web site redesign, so there will be plenty of additional detail on all things new SNF arriving over the next few weeks). Who does it exchange information with and how? Does it need special ports open? I'm going to assume this question is at least in part about security issues so I will frame my response from that perspective: Every minute or so (adjusted dynamically by the system) each new SNF node contacts one of our SYNC servers. The connection is made to port 25 by default. Since most MTAs will regularly make these kinds of connections no special ports will need to be open. If your MTAs are not allowed make outbound connections to port 25 for some reason then you have the option to make the connection to port 80. Our SYNC server software rejects connections by default. If an SNF node follows the expected connection protocols and authenticates properly and consistently then it will be allowed to communicate with the system. If it fails to do any of these things or looks suspicious in any way then it will be automatically black listed for increasingly extended periods and potentially null routed by our fire-walls. The security mechanisms are fully automatic and constantly monitored. The authentication protocol used to identify properly licensed SNF nodes is described in the file AuthenticationProtocol.swf. This file is included in the beta distributions and is also visible in the page linked below. At present there is no mechanism for connections to be made into SNF nodes -- only from SNF nodes to the SYNC servers. Also, there is no mechanism that allows the SYNC server (or any of our systems) to manipulate the SNF nodes except by the protocols described in the GBUdb documentation and by reporting update availability, tuning data, etc. This link helps explain how these interactions work: http://kb.armresearch.com/index.php?title=Message_Sniffer.TechnicalDetails.GBUdb The SYNC system is separate from the rulebase delivery system. All of the data transmitted and received by the SNF nodes is in plain text or base64 encoded. The format of the data is XML. With the exception of GBUdb traffic, the telemetry transmitted to us is available to you directly in the .status. reports made by the SNF engine. Status reports can be found in the same directory as your SNF log files. You can use these XML based posts to create your own real-time monitoring systems. In addition to the GBUdb functions, the telemetry eliminates the need to send in log files by providing near real-time pattern matching statistics; supports virtual spamtraps and other collaborative learning systems; and provides performance analysis, error detection / correction, and system tuning metrics. One other security note -- the virtual spamtrap system can be turned off easily if you wish. Normally the virtual spamtrap system will send us a random sampling of messages that come from the worst known IPs when those messages do not match known pattern rules. In most systems, these are messages that would normally be discarded. Samples are infrequent by design so they should not account for any appreciable bandwidth. Similarly, the GBUdb protocol is designed to share information sparsely so that no appreciable bandwidth or CPU capacity is required. Please let me know if I missed the mark on your questions. Hope this helps, _M -- # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: New SPAM pain
If Pete doesn't mind I will post my observations in regards to the product. I run both products (CommTouch and Sniffer). Darrell --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. John Shacklett writes: I'm dying to start a thread and talk about Sniffer's stance on CommTouch, but I can resist. Instead, I would like to point out that eight clearly spam messages have made it through to my Inbox [or Outlook Junk Folder] so far this week that appear to have skinned clear through Sniffer. First ones I've seen in Are we undergoing a new phase or campaign that I can make adjustments for? -- John # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: New SPAM pain
The more I think about it I am sorry about this post below - it kinda put's Pete on the spot - and I am sorry about that. Def. not my intention.. Darrell Darrell ([EMAIL PROTECTED]) writes: If Pete doesn't mind I will post my observations in regards to the product. I run both products (CommTouch and Sniffer). Darrell --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. John Shacklett writes: I'm dying to start a thread and talk about Sniffer's stance on CommTouch, but I can resist. Instead, I would like to point out that eight clearly spam messages have made it through to my Inbox [or Outlook Junk Folder] so far this week that appear to have skinned clear through Sniffer. First ones I've seen in Are we undergoing a new phase or campaign that I can make adjustments for? -- John # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] --- Check out http://www.invariantsystems.com for utilities for Declude, Imail, mxGuard, and ORF. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: ANN: Availability of 5xxSink 0.5.00, IIS SMTP event sink for text-file recipient validation
Sandy actually released an updated version that allows for that. http://www.mail-archive.com/declude.junkmail@declude.com/msg27158.html Darrell fpReview - Review held mail the easy way. http://www.invariantsystems.com - Original Message - From: Paul Fuhrmeister [EMAIL PROTECTED] To: Message Sniffer Community sniffer@sortmonster.com Sent: Wednesday, June 14, 2006 9:17 PM Subject: [sniffer] Re: ANN: Availability of 5xxSink 0.5.00, IIS SMTP event sink for text-file recipient validation Does anyone know of a similar solution that allows wild cards (allow anything at domain name). We still have some customers using catch-all accounts. Paul Fuhrmeister [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sanford Whiteman Sent: Saturday, December 03, 2005 6:29 AM To: Declude.JunkMail@declude.com; IMail_Forum@list.ipswitch.com; Declude.Virus@declude.com; sniffer@SortMonster.com Subject: [sniffer] ANN: Availability of 5xxSink 0.5.00, IIS SMTP event sink for text-file recipient validation All, I've posted 5XXSINK, an IIS SMTP event sink (freeware) that allows you to block unknown recipients at your IIS 5.0 or 6.0 MX by populating a barebones textfile. Those who use the powerful IIS SMTP engine as an MX have no built-in method of preventing brute-force spam runs from overwhelming their internal content scanning and mailbox servers with wasted message processing and double-bounce generation. Several commercial anti-abuse products can add this functionality, but they can add undue cost to a large server farm, and seem like overkill when a well-tuned content scanning engine (IMail's, Declude's, etc.) already exists internally, save for the fact that it sees messages that should never get that far. While it is debatable whether having envelope recipient validation at your MXs will reduce the number of spammers making initial connections to you, it cannot be denied that having such validation will save your hardware and bandwidth resources beyond the first part of the SMTP conversation. Recipient validation at the MX can make the difference between a workable anti-spam content scanner and one that fails because it's overwhelmed by messages it should never see. 5XXSINK is designed to do one thing, do it well, and do it for free: to look up full e-mail addresses in a locally stored text file and reject all RCPT TO commands that do match a line in the file. That's it. 5XXSINK is NOT designed to do any of the following: connection throttling, tarpitting, greylisting, sender validation, HELO interpretation, or DNSBL lookups. It expects that a robust content scanning solution exists behind, or perhaps on, the IIS SMTP server (although commercial IIS SMTP integrations solutions usually duplicate 5XXSINK's recipient validation functionality -- and then some). Again, the sole function is to keep messages that absolutely, positively do not need to be scanned out of the scanning path. There are no false positives with recipient validation, so it's an obvious first step in an anti-abuse chain. 5XXSINK is multithreaded and likely performs its very particular function as fast as practically possible. * * * Download: http://www.imprimia.com/products/software/freeutils/5xxsink/download/release Be sure to go over the README in-depth. That's where it's at. Support: Through the IMail and Declude support lists, as the communities primarily served by the product. Please post support questions as [OT] to create a public archive and to encourage knowledge sharing. --Sandy This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
Re: [sniffer] Watch out... SURBL SORBS full of large ISPs and Antispamprovidres.
Pete, I just checked real quick hitting several DNS servers (mine and others) and I am not seeing this - are you still seeing this now? C:\nslookup 2.0.0.127.multi.surbl.org Server: nscache5.bflony.adelphia.net Address: 68.168.224.180 Non-authoritative answer: Name:2.0.0.127.multi.surbl.org Address: 127.0.0.126 C:\nslookup declude.com.multi.surbl.org Server: nscache5.bflony.adelphia.net Address: 68.168.224.180 *** nscache5.bflony.adelphia.net can't find declude.com.multi.surbl.org: Non-exi stent domain C:\nslookup w3.org.multi.surbl.org Server: nscache5.bflony.adelphia.net Address: 68.168.224.180 *** nscache5.bflony.adelphia.net can't find w3.org.multi.surbl.org: Non-existent domain Darrell Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. - Original Message - From: Matt [EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Tuesday, January 17, 2006 7:21 AM Subject: Re: [sniffer] Watch out... SURBL SORBS full of large ISPs and Antispamprovidres. Pete, w3.org would be a huge problem because Outlook will insert this in the XML headers of any HTML generated E-mail. If you could give us an idea of when this started and possibly ended, that would help in the process of review. Thanks, Matt Pete McNeil wrote: Hello Sniffer Folks, Watch out for false positives. This morning along with the current spam storm we discovered that SURBL and SORBs are listing a large number of ISP domains and anti-spam service/software providers. As a result, many of these were tagged by our bots due to spam arriving at our system with those domains and IPs. Most IPs and domains for these services are coded with nokens in our system to prevent this kind of thing, but a few slipped through. We are aggressively hunting any more that might have arrived. You may want to temporarily reduce the weight of the experimental IP and experimental ad-hoc rule groups until we have identified and removed the bad rules we don't know about yet. Please also do your best to report any false positives that you do identify so that we can remove any bad rules. I don't expect that there will be too many, but I do want to clear them out quickly if they are there. Please also, if you haven't already, review the false positive procedures: http://www.sortmonster.com/MessageSniffer/Help/FalsePositivesHelp.html Pay special attention to the rule-panic procedure and feature in case you are one of the services hit by these bad entries. An example of some that we've found in SURBL for example are declude.com, usinternet.com, and w3.org It's not clear yet how large the problem is, but I'm sure it will be resolved soon. Hope this helps, Thanks, _M Pete McNeil (Madscientist) President, MicroNeil Research Corporation Chief SortMonster (www.sortmonster.com) Chief Scientist (www.armresearch.com) This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Rash of false positives
It is used in both versions for different things. Darrell ---Check out http://www.invariantsystems.com for utilities for Declude, mxGuard,and Imail. IMail Queue Monitoring, Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. - Original Message - From: Serge To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 9:27 PM Subject: Re: [sniffer] Rash of false positives i thought declude.cfg is for V 3.x Am I wrong ?is declude.cfg used with V 2.x ? - Original Message - From: John Moore To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 11:12 PM Subject: RE: [sniffer] Rash of false positives Matt, Thank you for your help and thorough explanation. I added the declude.cfg with the PROCESSES 20 We are running declude 2.06 and have the JM pro and AV standard. We will look into getting the persistent mode setup and see if that helps as well. Thanks, again. John From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MattSent: Wednesday, November 09, 2005 4:49 PMTo: sniffer@SortMonster.comSubject: Re: [sniffer] Rash of false positives John,The mystery heap issue is a memory issue with Windows where it only reserves so much memory for running things like Declude, Sniffer, other external tests and your virus scanners. If you have something that is hanging, running slowly, or taking too long, it can gobble up all of the memory available to these launched processes and then result in errors. Generally speaking, you can only get about 40 or so processes of these types to run at one time before you could start seeing these errors. Declude counts as one process, and often there is one other process that Declude launches that goes to this count (external tests and virus scanners are all run in serial so only one can be launched at a time by a single Declude process). If you have something like a virus scanner that crashes and then pops up a window on your next login, this can count towards the number of open processes.You can specify in Declude how many processes to run before Declude starts dumping things into an overflow, either the overflow folder in 2.x and before, or something under proc in 3.x. If you create a file called Declude.cfg and place in it "PROCESSES 20" that should protect you from hitting the mystery heap's limitations unless something is crashing and hanging. You might want to check Task Manager for processes to verify if things are hanging since not everything will pop up a window.I believe that running Sniffer in persistent mode will help to alleviate this condition, but it's only one part and if the mystery heap is the cause, it might just cause the errors to be triggered on other IMail launched processes including Declude.exe and your virus scanners.MattJohn Moore wrote: We have not run snf2check on the updates. And it may be a coincidence or bad timing that sniffer appears to be the culprit. But we have stopped sniffer (commented out in the declude global.cfg) for an observed period of time and the mail never stops (and had never stopped before sniffer) and conversely, it only stops when sniffer is running. We have not gone the extra steps of putting sniffer in persistent mode. We are looking at moving the imail/declude/sniffer setup to a newer box with more resources. Currently on a dell 2450 dual 833 and 1 gig of ram and raid 5. Volume of email is less than 10,000 emails per day. J From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Darin CoxSent: Wednesday, November 09, 2005 1:47 PMTo: sniffer@SortMonster.comSubject: Re: Re[4]: [sniffer] Rash of false positives Arecorrupted rulebase files the culprit? How do you update... and do you run snf2check on the updates? Just wondering if the rulebase file is theproblem, if the problemoccurs during the update, or if you are running into obscure errors with the EXE itself Darin. - Original Message - From: John Moore To: sniffer@SortMonster.com Sent: Wednesday, November 09, 2005 12:42 PM Subject: RE: Re[4]: [sniffer] Rash of false positives We had this same thing happen. It has been happening more frequently recently and we are looking into disabling sniffer as it seems to be the culprit each time. John Moore305 Spin
Re: [sniffer] Sniffer Resources
How does AVAFTERJM help? Unless you had JunkMail delete the message it would seem that it has to be scanned for viruses either way. This is more appropriate on the Declude list - but with that said you are correct in order for it to make a major impact you need to have actions that either hold, delete, or bounce for this to make a huge difference. I don't know which uses more processor time... Virus or SPAM scanning. If you use a bunch of tests it probably takes more horsepower to scan for SPAM than viruses. If that's the case then it would see like you would want to virus scan FIRST. Any message deleted by the virus scanner don't need to be scanned for SPAM. Couple of things to keep in mind. One such thing is virus scanner - if you use Mcafee it is VERY resource intensive so this helps alot. The other thing which complements the first point is that most normal servers have approx 95%+ spam volume and if you don't have to virus scan 95% of mail than it saves a lot of resources. Darrell --- Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail Queue Monitoring, Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Sniffer taking a long time?
What kind of box are you running Sniffer on and what is your CPU. I have found that most of the messages that get scanned on my system by Sniffer are done in under a second. Darrell Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Dan Horne writes: More info: When I stop the Sniffer service, processing time goes to milliseconds. Start the service back and it is back up to a minute. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Horne Sent: Monday, August 01, 2005 11:58 AM To: sniffer@SortMonster.com Subject: RE: [sniffer] Sniffer taking a long time? Here are the sniffer log entries for each of the messages, if that helps any: 08/01/2005 11:32:51.747 Q40a201cc1a59 SNIFFER: External program started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe mysnifferauthcode S:\imail\spool\D40a201cc1a59.SMD 08/01/2005 11:33:46.751 Q40a201cc1a59 SNIFFER: External program reports exit code of 61 20050801153252 D40a201cc1a59.SMD 70 20 Match 266707 61 343 358 50 20050801153252 D40a201cc1a59.SMD 70 20 Match 426427 61 1915192950 20050801153252 D40a201cc1a59.SMD 70 20 Final 266707 61 0 502050 08/01/2005 11:30:53.757 Q402b01b61a28 SNIFFER: External program started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe mysnifferauthcode S:\imail\spool\D402b01b61a28.SMD 08/01/2005 11:31:48.210 Q402b01b61a28 SNIFFER: External program reports exit code of 52 20050801153054 D402b01b61a28.SMD 80 10 Match 372669 52 2745286060 20050801153054 D402b01b61a28.SMD 80 10 Match 423177 61 2695303660 20050801153054 D402b01b61a28.SMD 80 10 Match 372652 61 2695313860 20050801153054 D402b01b61a28.SMD 80 10 Final 372669 52 0 495260 08/01/2005 11:30:56.561 Q402a01cc1a27 SNIFFER: External program started: M:\IMail\Sniffer2\Distribution\Winx\mysniffer.exe mysnifferauthcode S:\imail\spool\D402a01cc1a27.SMD 08/01/2005 11:31:51.074 Q402a01cc1a27 SNIFFER: External program reports exit code of 0 20050801153056 D402a01cc1a27.SMD 190 40 White 137999 0 2256228544 20050801153056 D402a01cc1a27.SMD 190 40 Final 137999 0 0 24419 44 This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Spam blocks loading me up with spam
Scott, Not to many incoming for me - about 200 out of about 125K messages. One thing to note is the ones I am getting are around that block but even lower like 200.49.44.x. Darrell ---Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail Queue Monitoring, Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. - Original Message - From: Scott Fisher To: sniffer@SortMonster.com Sent: Thursday, June 16, 2005 6:04 PM Subject: [sniffer] Spam blocks loading me up with spam Am I the only one getting blasted by these spam from these IP blocks? Sniffer seems a little behind on catching these. 200.49.48.0/24200.49.48.0/24 200.49.49.0/24200.49.49.0/24mowz2.com200.49.50.0/24200.49.50.0/24qckcstmr.com 200.49.51.0/24200.49.51.0/24srvdupfrsh.com200.49.52.0/24200.49.52.0/24aahtv.com200.49.53.0/24200.49.53.0/24aakai.com 200.49.54.0/24200.49.54.0/24aakib.com200.49.55.0/24200.49.55.0/24aakli.com200.49.56.0/24200.49.56.0/24aafix.com200.49.57.0/24200.49.57.0/24e.com 200.49.58.0/24200.49.58.0/24200.49.59.0/24200.49.59.0/24 Domain names andlinks seem to be five chars beginning with aa. Theyalsoseem to be progressing through theIP blocks. i think they started in on the June 15th and have been spamming pretty consistantly.
Re: [sniffer] mini-obfuscation
Pete,
Doesnt Sniffer have a certain level of support for regex's? I know we have
had good luck with regex's like this which catch obfuscation techniques with
viagra with Declude. We found it easier to use regex's than to list all of
the different variations.
(?:\b|\s)[_\W]{0,3}(?:\\\/|V)[_\W]{0,3}[ij1!|l\xEC\xED\xEE\xEF][_\W]{0,3}[a4
[EMAIL PROTECTED],3}[xyz]?[gj][_\W]{0,3}r[_\W]{0,[EMAIL PROTECTED],
3}x?[_\W]{0,3}(?:\b|\s)
Darrell
Check out http://www.invariantsystems.com for utilities for Declude And
Imail. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG
Integration, and Log Parsers.
Pete McNeil writes:
On Tuesday, March 22, 2005, 8:31:07 PM, Andrew wrote:
snip/
CA How many times have we all been frustrated that a piece of spam ending
CA up in *OUR* mailbox that was s close in content to spam we whacked
CA yesterday?
CA I thought the top n obfuscations might be interesting to look at, and
CA perhaps a shortcut (temporary, albeit) for spam catching. I thought we
CA might see whether, for example, broken URLs, fake comments, or high-bit
CA ASCII character substitutions were the obfuscation technique du jour.
Here you hit it IMHO. The reality appears to be, from my experience,
that small domains of obfuscation patterns rise and fall like swells
on the ocean. That is, stability tends to arise in one domain of
message characteristics and then fall to rise in another domain.
Sometimes the domain is well understood and sometimes it is entirely
new and forces us to think differently about what a feature really
is.
By domain I mean things like message structure, word obfuscation
techniques, phrase based swapping, html exploitation, etc...
The du jour part of your statement is a key element to the problem.
Defining and re-defining the conceptual framework that describes
feature domains in the spam is the other key element.
Put more simply - knowing what to look for is a basic element, but it
gets you nowhere on it's own. Knowing (recognizing) when to look for
the what is the key that makes the problem workable.
CA I while back curiousity got the better of me (it was raining, and
CA I had a few days off) and I did a few grep sweeps on a warm spam
CA corpus.
CA I was disappointed in my success rate for:
CA v.?i.?a.?g.?r.?a.?
CA and similar queries with deliberately substitutions (e.g. using a 1
CA for i). I started writing a grep-generating-permutation engine and
CA decided my time was better spent on scritching my cat under his chin.
That is a nifty direction that I wish I had more time for. Perhaps I
will some day soon when Sniffer get's slashdotted and sales go through
the roof!
--- meantime, back on this planet, I suggested a very similar thing to
Paul Graham at the first spam conference at MIT. As I recall he said
it was ambitious - a description that I have learned has a special
meaning in scientific circles. Something having to do with avian swine
and snowballs that have successful careers as tour guides in hell.
One of these days I think I might do it anyway, just to prove the
point, but in the mean time I too prefer to spend more time with my
cat. ;-)
Don't get me wrong - I strongly believe it can be done this way, but
it requires much more than good technology. It runs right into one of
the biggest problems with AI and, perhaps more importantly, people's
expectations of AI. No matter how good the pattern learning system
might be it will always lack the human experience. Computers don't
date or gain weight - so they have a hard time understanding what much
of the spam is about simply by looking at the patterns. That's why
the Message Sniffer process is designed with people tightly integrated
into the system.
_M
This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] OT - Microsoft Patch Day - Exchange and SMTP updates
The MS04-35 reissue some how slipped under the radar yesterday of the other patches.. So far no public exploits for that. However, SANS is indicating POC code has been released for MS05-05/09. So far for the cycle I patched one LOW volume production mail server and one standby server. Both of those are showing no issues. Anyone else apply these yet? Darrell Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, MRTG Integration, and Log Parsers. Colbeck, Andrew writes: Hello, all. Aside from the usual Internet Explorer and Office patches, this patch cycle also includes an update to the October update MS04-035 which affects a DNS query vulnerability in the SMTP handling in Windows 2000/2003 as well as Exchange 2003. http://www.microsoft.com/technet/security/bulletin/ms04-035.mspx This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
