[sniffer] Weak rule removal work...
Hello Sniffer Folks, I have been doing some work in the database today to make the rule strength analysis and weak rule removal process more efficient. Along the way I discovered an appreciable number of rules that had somehow been left with high strength numbers even though their recent activity values were zero. I have corrected this code. I expect that this will reduce the size of the rulebase files, though I am not yet certain how big the change will be. I am hopeful that the change will be large enough to yield a performance increase. There should be only positive impacts from the changes that I have made, but just in case I will be watching things very closely. Please let me know right away if you sense any drastic changes other than, perhaps, the size of the rulebase files. I've made arrangements to put everything back the way it was if need be ;-) Thanks, _M Pete McNeil (Madscientist) President, MicroNeil Research Corporation Chief SortMonster (www.sortmonster.com) This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Weak rule removal work...
Thanks Pete, these are the kind of proactive notification I wish some of our other vendors followed. Bill -Original Message- From: Pete McNeil [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 04, 2005 1:35 PM To: sniffer@SortMonster.com Subject: [sniffer] Weak rule removal work... Hello Sniffer Folks, I have been doing some work in the database today to make the rule strength analysis and weak rule removal process more efficient. Along the way I discovered an appreciable number of rules that had somehow been left with high strength numbers even though their recent activity values were zero. I have corrected this code. I expect that this will reduce the size of the rulebase files, though I am not yet certain how big the change will be. I am hopeful that the change will be large enough to yield a performance increase. There should be only positive impacts from the changes that I have made, but just in case I will be watching things very closely. Please let me know right away if you sense any drastic changes other than, perhaps, the size of the rulebase files. I've made arrangements to put everything back the way it was if need be ;-) Thanks, _M Pete McNeil (Madscientist) President, MicroNeil Research Corporation Chief SortMonster (www.sortmonster.com) This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html --- This message and any included attachments are from Siemens Medical Solutions USA, Inc. and are intended only for the addressee(s). The information contained herein may include trade secrets or privileged or otherwise confidential information. Unauthorized review, forwarding, printing, copying, distributing, or using such information is strictly prohibited and may be unlawful. If you received this message in error, or have reason to believe you are not authorized to receive it, please promptly delete this message and notify the sender by e-mail with a copy to [EMAIL PROTECTED] Thank you This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] new spam storm?
I've sure been seeing it. My db updates are triggered off email update notices from sniffer, so I know I have the latest. Feels like something's gone wrong with sniffer due to the year change. Rick Robeson getlocalnews.com [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Kirk Mitchell Sent: Tuesday, January 04, 2005 2:56 PM To: sniffer@SortMonster.com Subject: [sniffer] new spam storm? Seems like I've been getting a ton of spam in the last few days that's been scored as either LOW or CLEAN, many of them for cheap drugs, watches or my cheating wife. I have AutoSNF running every 2 hours, so it shouldn't be due to outdated rulesets. Is anyone else seeing this, or could I be missing something? Thanks, -- Kirk Mitchell-General Manager[EMAIL PROTECTED] Keystone Connect Unlock Your World Altoona, PA 814-941-5000 http://www.keyconn.net This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] new spam storm?
many of them for ... my cheating wife. Sorry to hear about your marital problems. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kirk Mitchell Sent: Tuesday, January 04, 2005 05:56 PM To: sniffer@SortMonster.com Subject: [sniffer] new spam storm? Seems like I've been getting a ton of spam in the last few days that's been scored as either LOW or CLEAN, many of them for cheap drugs, watches or my cheating wife. I have AutoSNF running every 2 hours, so it shouldn't be due to outdated rulesets. Is anyone else seeing this, or could I be missing something? Thanks, -- Kirk Mitchell-General Manager[EMAIL PROTECTED] Keystone Connect Unlock Your World Altoona, PA 814-941-5000 http://www.keyconn.net This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] new spam storm?
I've noted that dictionary attack type spam is generally of this variety, and while you are probably blocking a great deal of this, the sheer volume makes it look like you aren't doing that well against it. I've also noted that the domains that they use are frequently changed, thus escaping both SURBL and Sniffer for periods of time. I am under the impression that these spammers have taken to using multiple domains at once and segmenting the domains that they attack with them so that if one domain gets listed in SURBL (or Sniffer for a select group), then it won't affect their entire campaign. Some of these campaigns are so high in volume that there is no way that the domains could otherwise escape being listed for more than 15 minutes. This technique would fall under the guise of if I was a spammer, this would be what I would do. Generally these guys are only underachievers because spam prevention generally sucks and even if blocked, the anti-social characteristics of hijacking computers and pummeling others with their garbage has enough redeeming value (from their perspective) to keep them happy. They are however capable of finding ways around almost every method that we use, but they for the most part just don't bother to try, but they are definitely trying harder than before. Something else that I have noted recently is that they seem to be going after DUL space overseas instead of exclusively crawling well known and well tagged IP space in North America. It seems that the majority of zombie generated spam that gets through or is scored low on my system is originating from overseas. Maybe applicable in your case, maybe not. I believe that Pete's plans for incremental updates will help to address such issues by making Sniffer even more real-time than it already is. Matt Kirk Mitchell wrote: Seems like I've been getting a ton of spam in the last few days that's been scored as either LOW or CLEAN, many of them for cheap drugs, watches or my cheating wife. I have AutoSNF running every 2 hours, so it shouldn't be due to outdated rulesets. Is anyone else seeing this, or could I be missing something? Thanks, -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[2]: [sniffer] new spam storm?
On Tuesday, January 4, 2005, 6:06:00 PM, Rick wrote: RR I've sure been seeing it. My db updates are triggered off email update RR notices from sniffer, so I know I have the latest. RR Feels like something's gone wrong with sniffer due to the year change. We are definitely experiencing a spam storm - showing new rules at the rate of 745 yesterday and we're on track for at least the same number today. Also, during the holidays we almost stopped getting user submissions - so we're behind on some things that we haven't seen yet. I'm not seeing any indication of any problems - in particularly none related to the year change. SNF doesn't pay any particular attention to the date - only the patterns it can match. Be sure to submit what gets through. If you think everything is getting through then check for errors in your SNF file. Hope this helps, _M PS: To reference the rate of new spam rules being added see this URL: http://www.sortmonster.com/MessageSniffer/Performance/ChangeRates.jsp This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] new spam storm?
At 06:03 PM 1/4/2005 -0500, Andy Schmidt wrote: many of them for ... my cheating wife. Sorry to hear about your marital problems. LOL! Apparently the tramp's been sleeping all over, and there are plenty of websites that can show me how, where, when, and with whom. Darned if I know when she's had time to do it though. I haven't clicked on any of the links though, you think maybe it's because I'm afraid that the pic on the site is really her? -- Kirk Mitchell-General Manager[EMAIL PROTECTED] Keystone Connect Unlock Your World Altoona, PA 814-941-5000 http://www.keyconn.net This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[2]: [sniffer] new spam storm?
On Tuesday, January 4, 2005, 6:13:24 PM, Matt wrote: M I've noted that dictionary attack type spam is generally of this M variety, and while you are probably blocking a great deal of this, the M sheer volume makes it look like you aren't doing that well against it. M I've also noted that the domains that they use are frequently changed, M thus escaping both SURBL and Sniffer for periods of time. I am under M the impression that these spammers have taken to using multiple domains M at once and segmenting the domains that they attack with them so that if M one domain gets listed in SURBL (or Sniffer for a select group), then it M won't affect their entire campaign. Some of these campaigns are so high M in volume that there is no way that the domains could otherwise escape M being listed for more than 15 minutes. snip/ M I believe that Pete's plans for incremental updates will help to address M such issues by making Sniffer even more real-time than it already is. These are interesting comments... yeah - when one of these new campaigns gets started, if there is a hole you can get pounded by hundreds of messages before the hole is closed - making it seem like the dam has broken. One thing we are doing about these campaigns is coding not only URI, but text segments, abstract patterns, and increasingly I've spent time creating compound message structure rules --- so that when they swap out text, images, uri and other variable components the message can still be captured. I know from my monitoring that I've made a dent in this stuff this way - but there is definitely a lot of it and more work needs to be done to find and respond w/ message structure rules. As for near real-time incremental updates, that is planned, but it is definitely off in the future. Hopefully before mid-year though. _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] RuleBase ktk82hrr
Dear Pete, Our rulebase file grew from 11 meg to 17.5 meg since the last download a few hours ago. Is this right? Michael Stein Computer House [EMAIL PROTECTED] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] RuleBase ktk82hrr
Correction, make that 23 meg! Mike - Original Message - From: Computer House Support [EMAIL PROTECTED] To: sniffer@SortMonster.com Sent: Wednesday, January 05, 2005 12:33 AM Subject: [sniffer] RuleBase ktk82hrr Dear Pete, Our rulebase file grew from 11 meg to 17.5 meg since the last download a few hours ago. Is this right? Michael Stein Computer House [EMAIL PROTECTED] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[2]: [sniffer] RuleBase ktk82hrr
On Wednesday, January 5, 2005, 12:41:34 AM, Computer wrote: CHS Correction, make that 23 meg! Thanks for the heads up --- something is wrong, I'll figure it out. You compiled with 231000 rules! _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Rule strength tuning gone crazy.
Hello Sniffer Folks, The changes in the rule strength tuning have uncovered a bug in the rulebase compilers. The result of this bug was that shortly after removing approximately 7 weak rules, the rulebase began to compile with nearly twice as many rules as they should have (213000+). This problem was discovered and all changes were reversed around 0100 EST. All rulebases are now recompiling with the original settings and should be _normal_ again within 6-8 hours. Once the problem with the tuning engine on the compilers has been corrected, the changes to the rule strength analysis system will be re-tried. The result of these events _should_ only be that your rulebase file may be significantly larger than usual for a short period of time. There should be no other effects except perhaps a slightly higher system load during the period of time that the rulebase file is oversized. This is _NOT_ a problem with the SNF software that runs on your servers. The problem is with the software we use to create rulebase files. I expect things to be normal shortly and corrected within a day so that we can proceed with the tuning improvements. Sorry for any confusion this may have caused. Thanks, _M Pete McNeil (Madscientist) President, MicroNeil Research Corporation Chief SortMonster (www.sortmonster.com) This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] RuleBase ktk82hrr
Yep, just checked mine rulebase too, went from 17mb to just under 25mb. Things still appear to be functioning okay. Bill -Original Message- From: Pete McNeil [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 04, 2005 9:49 PM To: Computer House Support Subject: Re[2]: [sniffer] RuleBase ktk82hrr On Wednesday, January 5, 2005, 12:41:34 AM, Computer wrote: CHS Correction, make that 23 meg! Thanks for the heads up --- something is wrong, I'll figure it out. You compiled with 231000 rules! _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html --- This message and any included attachments are from Siemens Medical Solutions USA, Inc. and are intended only for the addressee(s). The information contained herein may include trade secrets or privileged or otherwise confidential information. Unauthorized review, forwarding, printing, copying, distributing, or using such information is strictly prohibited and may be unlawful. If you received this message in error, or have reason to believe you are not authorized to receive it, please promptly delete this message and notify the sender by e-mail with a copy to [EMAIL PROTECTED] Thank you This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
