Re[4]: [sniffer] Sniffer and SURBL
On Monday, January 10, 2005, 8:50:37 PM, Andrew wrote: CA> Thanks, Pete. CA> I was thinking that Sniffer's l33t ninja skillz would be well-used for CA> searching a large corpus of URIs, particularly the current bout of CA> spammers you and I mentioned before Xmas (the ones that are specifying CA> the domain name, not a URL, and which Sniffer is catching because of the CA> consistent instructions, regardless of the dynamically changing domain CA> names), as a URI filter might miss them because of obfuscation, or might CA> miss the real payload. Sniffer would catch these URIs, because it only CA> cares about tokenized text, not whether that text was detected in a URL. CA> There would still be a place for both SURBL lookups and Sniffer in that CA> scenario, because they are refreshed on different schedules and have CA> independent spamtraps feeding them. CA> I wasn't thinking about Sniffer incorporating a real-time lookup; I CA> agree with your direction for the product. For the reason you cited, CA> I'll go a little further and say that Sniffer would have to really break CA> out in a new direction to be worth implementing a real-time lookup of CA> some sort. I agree. Thanks for clarifying. The only real-time stuff we have planned is proactive -- where trusted peers and our control nodes will share some real time data eventually. With regard to incorporating SURBL in SNF... I wonder about that for a lot of reasons. With a sufficient hardware it would be possible to fold in SURBL and a number of other services (SBL for example) - though the rulebase would be quite large I'm sure, and I think I would prefer to have those rulebases separate. Perhaps there's a future solution in that statement -- perhaps someday other public BL systems might be automatically incorporated in to SNF readable distributions -- though with the exception of URI based systems I think DNS distribution mechanisms are probably the best choice. I'll have to keep thinking about this. Thanks! _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Sniffer and SURBL
Pete, Not that I necessarily expect for this to happen, but rather something to consider as things progress... With the cross-checking of SURBL data, one needs to be careful to not be double scoring tests that target the same piece of data. An example of this would be scoring each DUL hit separately as opposed to scoring DUL hits, one or many, as a single scoreable hit. Unfortunately we don't know exactly what triggers a hit in Sniffer when it happens. In reality, the very high correlation between SURBL hits and Sniffer can be attributed to both the +95% spam hit rates in Sniffer, but also the cross referencing. If I were to be able to tell that Sniffer hit on some other form of content, and SURBL hit on the URL, the combination of hits would be stronger as a group, but not knowing, and understanding that there would be a fair amount of hits for the same piece of data, the combination of hits should be treated as weaker as a group than the sum of scores. I guess the only way to modify Sniffer to such needs might be to reclassify rules based on the type of data that the rule targets instead of the type of content the rule was generated from. As you are aware, Subject rules are weaker than body domain name hits, and body domain name hits are weaker than full URL hits. Some current result codes are highly suggestive of the types of rules that are contained, such as obfuscation rules, but porn rules are rather wide open. I guess as an administrator, I would prefer to know the classification based on the reliability of the data used as opposed to the genre of spam from which the rule was created (and this is not perfectly consistent in subsequent hits). I fear that this would mostly benefit power users who construct combination filters from this data and would benefit from classifying such hits, though some benefit could come by way of the simple weighting of Sniffer in isolation from other such things where there are notable differences in the reliability of the data. This might also be somewhat impractical, and certainly not expected outside of a large change in the way that the app behaved. So again, just something to chew on, and I'm sure it has crossed your mind before. Thanks, Matt Pete McNeil wrote: On Monday, January 10, 2005, 7:17:29 PM, Andrew wrote: CA> Pete, I thought that you had said at one point that SortMonster fetches CA> one or more SURBL zones and incorporates those as spam data for Message CA> Sniffer? CA> It seems like a great idea to me. But then, from my distance, a lot of CA> things look like a good idea for someone else to implement! That's not exactly how it works - What we do is that our robots will look at some of the messages that hit our spamtraps and if they find a URI that looks like a good choice they will cross check it with SURBL. More often than not we've already got the URI coded from our manual work, but this robotic mechanism allows the rulebase to keep up minute by minute - and since the email triggering this work has come in through one of our spamtraps, it acts like an extra check - so those listings that we do have tend to be very solid. At some point we may bolt on some additional real-time lookups like SURBL etc... but we don't have plans for that just yet, and most installations already have these tools employed in other mechanisms they are running, so it would be redundant for us to add it - at least at this point. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Sniffer and SURBL
Thanks, Pete. I was thinking that Sniffer's l33t ninja skillz would be well-used for searching a large corpus of URIs, particularly the current bout of spammers you and I mentioned before Xmas (the ones that are specifying the domain name, not a URL, and which Sniffer is catching because of the consistent instructions, regardless of the dynamically changing domain names), as a URI filter might miss them because of obfuscation, or might miss the real payload. Sniffer would catch these URIs, because it only cares about tokenized text, not whether that text was detected in a URL. There would still be a place for both SURBL lookups and Sniffer in that scenario, because they are refreshed on different schedules and have independent spamtraps feeding them. I wasn't thinking about Sniffer incorporating a real-time lookup; I agree with your direction for the product. For the reason you cited, I'll go a little further and say that Sniffer would have to really break out in a new direction to be worth implementing a real-time lookup of some sort. Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, January 10, 2005 4:58 PM To: Colbeck, Andrew Subject: Re[2]: [sniffer] Sniffer and SURBL On Monday, January 10, 2005, 7:17:29 PM, Andrew wrote: CA> Pete, I thought that you had said at one point that SortMonster CA> fetches one or more SURBL zones and incorporates those as spam data CA> for Message Sniffer? CA> It seems like a great idea to me. But then, from my distance, a lot CA> of things look like a good idea for someone else to implement! That's not exactly how it works - What we do is that our robots will look at some of the messages that hit our spamtraps and if they find a URI that looks like a good choice they will cross check it with SURBL. More often than not we've already got the URI coded from our manual work, but this robotic mechanism allows the rulebase to keep up minute by minute - and since the email triggering this work has come in through one of our spamtraps, it acts like an extra check - so those listings that we do have tend to be very solid. At some point we may bolt on some additional real-time lookups like SURBL etc... but we don't have plans for that just yet, and most installations already have these tools employed in other mechanisms they are running, so it would be redundant for us to add it - at least at this point. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[2]: [sniffer] Sniffer and SURBL
On Monday, January 10, 2005, 7:17:29 PM, Andrew wrote: CA> Pete, I thought that you had said at one point that SortMonster fetches CA> one or more SURBL zones and incorporates those as spam data for Message CA> Sniffer? CA> It seems like a great idea to me. But then, from my distance, a lot of CA> things look like a good idea for someone else to implement! That's not exactly how it works - What we do is that our robots will look at some of the messages that hit our spamtraps and if they find a URI that looks like a good choice they will cross check it with SURBL. More often than not we've already got the URI coded from our manual work, but this robotic mechanism allows the rulebase to keep up minute by minute - and since the email triggering this work has come in through one of our spamtraps, it acts like an extra check - so those listings that we do have tend to be very solid. At some point we may bolt on some additional real-time lookups like SURBL etc... but we don't have plans for that just yet, and most installations already have these tools employed in other mechanisms they are running, so it would be redundant for us to add it - at least at this point. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Sniffer and SURBL
Pete, I thought that you had said at one point that SortMonster fetches one or more SURBL zones and incorporates those as spam data for Message Sniffer? It seems like a great idea to me. But then, from my distance, a lot of things look like a good idea for someone else to implement! Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, January 10, 2005 12:01 PM To: Phillip Cohen Subject: Re: [sniffer] Sniffer and SURBL On Monday, January 10, 2005, 3:05:18 PM, Phillip wrote: PC> How do you use both Sniffer and SURBL together? What else is PC> required. On most platforms SNF is integrated through, or in front of other anti-spam / anti-virus software. For example, SNF is frequently placed in front of SpamAssassin, or integrated with IPswitch products through Declude or mxGuard. These tools now include a way to leverage SURBL and other URI based blocking lists. There is no way for SNF to directly tie into SURBL or any other blocking list at this time. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] invURIBL (SURBL URI Query) Important Notice On Beta 4
After working with Andy we found that a DLL was missing out of our beta 4 package. If you downloaded invURIBL for the first time as Beta 4 you may be missing a DLL that was not included in that specific package. This DLL is used for additional mime decoding. I would recommend that everyone who downloaded Beta 4 please do so again as the package has been corrected to include the DLL. - http://www.invariantsystems.com/invuribl/default.htm We also recommend if you are using invURIBL that you please join the list serve for the application to stay informed on its status ([EMAIL PROTECTED]). Again, I am sorry for the inconvenience. Darrell Andy Schmidt writes: Hi Phil: A) I just corrected a bug in the setup of invURIBL (which is used to test again SURBL). I don't know yet what the impact is - but I BELIEVE that bug had caused more messages to be tagged than should have. Thus, the results were skewed, but I won't know until tomorrow by what degree. Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, MRTG Integration, and Log Parsers. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Sniffer and SURBL
Hi Phil: A) I just corrected a bug in the setup of invURIBL (which is used to test again SURBL). I don't know yet what the impact is - but I BELIEVE that the bug had caused more messages to be tagged than should have. Thus, the results were skewed, but I won't know until tomorrow by what degree. B) To run both tests from Declude do something like this: # set a "base" weight for Sniffer SNIFFER externalnonzero "sniffer.exe password" 1 0 # add extra weight to Sniffer for certain tests SNIFFER-SCAMS external053 "sniffer.exe password" 2 0 SNIFFER-PORNexternal054 "sniffer.exe password" 2 0 SNIFFER-MALWARE external055 "sniffer.exe password" 3 0 SNIFFER-OBFUSC external062 "sniffer.exe password" 2 0 # set a "base" weight for SURBL INV-URIBL external nonzero "INVURIBL.exe multi.surbl.org 20 %WEIGHT%" 0 0 # combine SURBL and Sniffer - if either exists incrase the weigth by 6 CONTENT filter D:\IMail\Declude\CONTENTfilter.txt x 6 0 C) The CONTENTfilter.txt file makes sure that there is no "double weight" when both Sniffer AND SURBL trigger at the same time: SKIPIFWEIGHT 20 TESTSFAILED 0 CONTAINSSNIFFER TESTSFAILED 0 CONTAINSINV-URIBL Best Regards Andy Schmidt H&M Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Phillip Cohen Sent: Monday, January 10, 2005 03:05 PM To: sniffer@SortMonster.com Subject: [sniffer] Sniffer and SURBL How do you use both Sniffer and SURBL together? What else is required. Phil This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Sniffer and SURBL
On Monday, January 10, 2005, 3:05:18 PM, Phillip wrote: PC> How do you use both Sniffer and SURBL together? What else is required. On most platforms SNF is integrated through, or in front of other anti-spam / anti-virus software. For example, SNF is frequently placed in front of SpamAssassin, or integrated with IPswitch products through Declude or mxGuard. These tools now include a way to leverage SURBL and other URI based blocking lists. There is no way for SNF to directly tie into SURBL or any other blocking list at this time. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Sniffer and SURBL
How do you use both Sniffer and SURBL together? What else is required. Phil This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Spam Ratios, specifically: Sniffer and SURBL
Hi Matt, Well, statistics are a tricky thing. When you had posted on the Sniffer or Declude lists over the weekend that I should provide more specific numbers, I had no yet understood how you calculated your "percent of SPAM". The key is always how one defines 100%. Now that I read your post on the Sniffer list, I realize what number you are looking for. You call it "percent of SPAM", I call it "percent of HELD messages" (which, in reality, is only a subset of all Spam.) Total Messages Processed: 13,077 Messages That Failed ANY Test(s): 11,323 (86.59%) Total Messages DELETE, HOLD, BOUNCE, ROUTETO: 7,737 (59.16%) Average Message Weight: 22.00 (Hold weight is 10) Note: Before anyone jumps down my throat for the low "hold" ratio, we simply whitelist a lot of internal mail based on SMTP AUTH and based on clients' static IPs. Of those 7,737 spam messages: INV-URIBL...7,737...59.16% IPNOTINMX...7,620...58.27% SNIFFER.7,396...56.56% -> or 95% of the messages that were "held" (which, matches your "capture" rate of 95 - 96% exactly!) Note: As stated in my original post, I ran additional reports to break out the unique hits by SURBL vs. Sniffer, vs. the combined hits. From that I concluded that SURBL is NOT an irrelevant subset of Sniffer - but rather there is about an 80 - 90% overlap. On both ends there are messages that one flags - but not the other. Thus my previous statement that by using both together I'm able to "tag" about 10 - 20% more messages than what each one individually would have tagged (tapping into the 40.84% of non-held messages). NOLEGITCONTENT..7,215...55.17% SPAMCOP.4,611...35.26% XBL-DYNA4,228...32.33% SORBS...4,221...32.28% DSBLSINGLE..3,686...28.19% REVDNS..2,967...22.69% WEIGHTFILTER2,841...21.73% SORBS-DUHL..2,436...18.63% HELOBOGUS...2,277...17.41% SENDERDB-BLOCK..2,095...16.02% SPAMROUTING.1,977...15.12% NJABLDYNA...1,958...14.97% DYNAMIC-IP..1,486...11.36% SPAMHEADERS.1,442...11.03% AHBL1,424...10.89% BLITZEDALL..1,359...10.39% NJABLPROXIES1,342...10.26% BCC41,313...10.04% SORBS-WEB...1,0267.85% BCC6..9277.09% BADHEADERS9267.08% AHBLPROXIES...9237.06% SBL...9187.02% SPAMDOMAINS...8346.38% SURBL-FROM7986.10% OPEN-RELAY7335.61% SORBS-HTTP7045.38% SNIFFER-PORN..6985.34% BCC8..6685.11% SORBS-SOCKS...6254.78% AHBLSOURCES...4913.75% RHSBL.3772.88% AHBLDOMAINS...2932.24% SPFFAIL...2762.11% SPFPASS...2351.80% BASE641871.43% MAILFROM..1821.39% NJABLDUL..1791.37% SENDERDB-SUSP.1451.11% SNIFFER-SCAMS.1110.85% FORMMAIL...850.65% NJABLSOURCES...710.54% SORBS-BADCONF..550.42% COMMENTS...410.31% SORBS-MISC.410.31% SNIFFER-MALWARE380.29% MULTI-RELAY330.25% DSBLMULTI..330.25% WEB-O-TRUST260.20% SORBS-ZOMBIE...230.18% SORBS-SMTP.230.18% MAILPOLICE-PORN220.17% SNIFFER-OBFUSC.150.11% ORDB...100.08% RDNSBL..50.04% NJABLRELAYS.50.04% HIL.40.03% Best Regards Andy Schmidt H&M Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Monday, January 10, 2005 11:35 AM To: sniffer@SortMonster.com Subject: Re: [sniffer] Still having problems I just wanted to add some stats that I thought might be of some use here. I gathered info on my block rates over the past three days and compared my Sniffer hits to them. There has been no measurable change to my system with an average of 96% of spam getting tagged by Sniffer. I'm at least not seeing any issues. FRIDAY == Blocked:89.45% of Total Message Volume Sniffer: 85.74% of Total Message Volume - Sniffer Capture Rate on Spam: 95.85% SATURDAY == Blocked:96.57% of Total Message Volume Sniffer: 92.55% of Total Message Volume - Sniffer Capture Rate on Spam: 95.84% SUNDAY ==
Re[2]: [sniffer] Still having problems
On Monday, January 10, 2005, 11:34:44 AM, Matt wrote: M> I just wanted to add some stats that I thought might be of M> some use here. I gathered info on my block rates over the past M> three days and compared my Sniffer hits to them. There has been no M> measurable change to my system with an average of 96% of spam M> getting tagged by Sniffer. I'm at least not seeing any issues. Thanks for all of this. I don't think there are any SNF issues - save a current spam storm: 545 new rule already and the day is very young! Some systems see more spam than others, have special needs, or a lower tolerance for leakage. The bursting order of spam sources changes constantly -- so the spam received at any given system can at times see sudden bursts of new spam activity with no apparent cause. This can also happen any time a given system begins to receive new spam sufficiently early when compared to when we see it, or when an update can go out. There are many mechanisms like this in play all the time and they give rise to random peaks and valleys in spam flow on every system -- often these events go un-noticed, but occasionally it can seem as though the dam has burst. In this case I think that a combination of things are happening. Thankfully - a failure in the SNF system doesn't appear to be one of them. _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: [sniffer] Still having problems
I just wanted to add some stats that I thought might be of some use here. I gathered info on my block rates over the past three days and compared my Sniffer hits to them. There has been no measurable change to my system with an average of 96% of spam getting tagged by Sniffer. I'm at least not seeing any issues. FRIDAY == Blocked: 89.45% of Total Message Volume Sniffer: 85.74% of Total Message Volume - Sniffer Capture Rate on Spam: 95.85% SATURDAY == Blocked: 96.57% of Total Message Volume Sniffer: 92.55% of Total Message Volume - Sniffer Capture Rate on Spam: 95.84% SUNDAY == Blocked: 96.19% of Total Message Volume Sniffer: 92.60% of Total Message Volume - Sniffer Capture Rate on Spam: 96.26% The way that I generated these stats was to assume that my "Hold" weight in Declude was an accurate approximate delineation between ham and spam. Then the total for the Sniffer tests was added together and divided by the block rate in order to calculate the "Sniffer Capture Rate on Spam". Hope this helps. Matt Pete McNeil wrote: On Monday, January 10, 2005, 12:38:45 AM, Kirk wrote: KM> I would like to attack this more aggressively. The increase we've seen in KM> spam getting through over the last week has brought on a dramatic increase KM> in customer complaints. What different approaches might I be able to take? I'm sorry to hear that. Spam is an increasing problem. I have adjusted your rulebase to the new rule strength threshold 0.5. Earlier today I coded a number of rules that are based on some of the subjects you submitted. If you can think of any black rules that you would feel comfortable coding on your system please let me know and I will add them. For example, you may be willing to accept single words or word pairs that we could not normally code into the core rulebase. I am open to any ideas you have and I will help you to create rules that meet your criteria. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
