Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.
Should be able to use "anonymous" as theusername and your e-mail address as password. Bill - Original Message - From: Gene Head To: [EMAIL PROTECTED] Sent: Thursday, December 25, 2003 9:56 PM Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. The site is asking for a username and password. Gene HeadACCRAM Inc.MCP,Net+,A+,CCNA,CCDA[EMAIL PROTECTED][EMAIL PROTECTED] -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kami RazvanSent: Tuesday, December 23, 2003 7:17 AMTo: [EMAIL PROTECTED]Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. Hi; The filters are available for anyone who wishes to use it.. the challenge is to keep this link out of the hands of search engines. Imagine the keywords that our our company will be associated with. If you want to use the filters simply visit the ftp site: ftp://ftp.OUR DOMAIN/IMail OUR DOMAIN = ClickandPledge.com These filters are updated 4 times a day. There are also some timed filters. Like Blacklists- you can choose 30 days, 10 days, etc. the same goes with the URL in body filters. We soon will have the timed BlacklistinBody filters. Please make sure to read the README.txt file first. Regards, Kami From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED]Sent: Tuesday, December 23, 2003 8:27 AMTo: [EMAIL PROTECTED]Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. Can we get a link to Kami's filters? Thanks. Neal Mathews Network Systems Engineer The Carriage House Co.'s, Inc. [EMAIL PROTECTED] wrote: - To: [EMAIL PROTECTED]From: "Matt Robertson" [EMAIL PROTECTED]Sent by: [EMAIL PROTECTED]Date: 12/22/2003 06:13PMSubject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. I came into this thread late, so my comments may not be strictly on point, but it seems to me the solution to this is to only use filters that work. Duh, right? In other words let the community validate and update Filter X and you simply plug in what you please.That means a centralized filter storage, update and distribution site. We actually aren't so far off that mark now. Look at Kami Razvan's ftp site and you'll find a treasure trove of filters there. A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today.I implemented most of Kami's stuff last week (supplementing most of the filters already installed that came from Matt Bramble and the result is a massive surge in my attach-to-kill ratio (on the kill side). There are so many I had to aggressively reorganize my global.cfg, but the results have been splendid, with the most processor-intensive filters not kicking in unless needed.I wrote a ColdFusion routine that downloads my selected filters, alters them to suit my skip and max weights, and uploads them to my mail server (the filters are regularly updated). Anyone who wants a copy let me know.-Matt Robertson, [EMAIL PROTECTED]MSB Designs, Inc. http://mysecretbase.com[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]---This E-mail came from the Declude.JunkMail mailing list. Tounsubscribe, just send an E-mail to [EMAIL PROTECTED], andtype "unsubscribe Declude.JunkMail". The archives can be foundat http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.
The site is asking for a username and password. Gene Head ACCRAM Inc. MCP,Net+,A+,CCNA,CCDA [EMAIL PROTECTED] [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kami Razvan Sent: Tuesday, December 23, 2003 7:17 AM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. Hi; The filters are available for anyone who wishes to use it.. the challenge is to keep this link out of the hands of search engines. Imagine the keywords that our our company will be associated with. If you want to use the filters simply visit the ftp site: ftp://ftp.OUR DOMAIN/IMail OUR DOMAIN = ClickandPledge.com These filters are updated 4 times a day. There are also some timed filters. Like Blacklists- you can choose 30 days, 10 days, etc. the same goes with the URL in body filters. We soon will have the timed BlacklistinBody filters. Please make sure to read the README.txt file first. Regards, Kami From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Tuesday, December 23, 2003 8:27 AM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. Can we get a link to Kami's filters? Thanks. Neal Mathews Network Systems Engineer The Carriage House Co.'s, Inc. [EMAIL PROTECTED] wrote: - To: [EMAIL PROTECTED] From: Matt Robertson [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] Date: 12/22/2003 06:13PM Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. I came into this thread late, so my comments may not be strictly on point, but it seems to me the solution to this is to only use filters that work. Duh, right? In other words let the community validate and update Filter X and you simply plug in what you please. That means a centralized filter storage, update and distribution site. We actually aren't so far off that mark now. Look at Kami Razvan's ftp site and you'll find a treasure trove of filters there. A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today. I implemented most of Kami's stuff last week (supplementing most of the filters already installed that came from Matt Bramble and the result is a massive surge in my attach-to-kill ratio (on the kill side). There are so many I had to aggressively reorganize my global.cfg, but the results have been splendid, with the most processor-intensive filters not kicking in unless needed. I wrote a ColdFusion routine that downloads my selected filters, alters them to suit my skip and max weights, and uploads them to my mail server (the filters are regularly updated). Anyone who wants a copy let me know. -- --- Matt Robertson, [EMAIL PROTECTED] MSB Designs, Inc. http://mysecretbase.com --- -- --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today. I am getting there. I know how it will be done, just need the time to set up the site. It will be accessible by HTTP and FTP. I am hoping by the weekend. John Tolmachoff Engineer/Consultant/Owner eServices For You --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
I, for one, will definitely pass on a central repository George, the way I am going to be setting it up will make it easy to view what ever filters some one wants to share, and then you pick and choose which ones you want to use. You can then get those files via ftp. I am also going to set up a list specifically for this, so that if any one say changes the format or such of a file, they can announce it. It will also be used to discuss the various files. John Tolmachoff Engineer/Consultant/Owner eServices For You --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.
Matt, Here are two analyses. The 11-15 to 11-30 covers the period from when I implemented your filters until I began using SKIPIFWEIGHT and MAXWEIGHT which obviously has some effect on the stats. The 11-15 to 12-21 expands the prior set to include the additional filters. There's also the weighting effect to consider. While I run the OBFUSCATION and Y!DIRECTED at hold weight (15), I use the GIBBERISH like the COMMENTS test and accumulate weight per hit. Since my SKIPIFWEIGHT is set to my DELETE weight (60), the filters will run until that's reached. These stats aren't a big deal to produce since its all in a SQL database. I'll be implementing your new filter versions this coming weekend (with new names to avoid commingling stats). I do strip out comments since they become meaningless as the filter contents are resequenced by my system. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 10:32 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. George, I think that logic can get you 95% of the way there with something as convoluted as this, that is run only about 1/3 of the time, and considering that you are only battling for about 2% of the processing power required by this filter alone, which shouldn't be too terribly much. Removing the comment blocks would probably have a bigger effect :) Changing to the new version of the filter should definitely help, though this isn't by far my most weighty filter. Here's something that I've very curious about though...the Y!DIRECTED filter contains a bunch of BODY searches for obfuscated strings, something that is almost totally redundant with the OBFUSCATION filter. I would be very curious to see how often those lines are hit because they could be dumped for a measurable performance increase. Any chance you want to take a crack at that? I wouldn't be surprised to see them never hit. Matt George Kulman wrote: Matt, I use LOGLEVEL HIGH for my data collection and analysis stuff and, as Bill pointed out, all hits are reflected. I've started to use SKIPIFWEIGHT. The result of course is that filters are bypassed and the statistics are skewed. For example on Friday 12/19, 15291 emails were processed by Declude on my system. Only 4604 were processed by the GIBBERISH filter. Of these 1328 had a total of 3854 hits. My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. If I collect all of the data, then I gain no benefit, since all of the processing takes place. If I take advantage of the analysis data, I reduce my processing workload but effectively destroy the validity of the statistical data which is now skewed by my filtering control. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 3:17 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. George, That's good data to have. I would have to assume that something tagged as gibberish in the main test would be random, and that's fairly well indicated by the somewhat tight range of the two character strings. Unless you are using a logging feature that I'm not aware of, you are only showing the last hit that the filter produces, and that explains why the Z strings are mostly bunched at the top. I've got these ordered alphabetically and will probably leave them there for management purposes. The counterbalances though are definitely something that I will use your information for reordering them. I believe I made an attempt to order these in the 2.0 filter version according to what I thought would be more common as well as what would be a faster search (BODY searches are slower than other things and will go lower in general, though a BODY search for base64 goes at the top because it is fairly common). Because of this and along with the above mentioned issue, the hit stats therefore aren't a perfect indication of what would save the most processing power, but it definitely helps if you just make some assumptions. I hadn't gathered any stats myself on the Auto-generated Codes that I added in about a month or so ago, and it's nice to see that they're getting hit since I was really just brainstorming about what types of things might be seen. I might remove some entries though if they aren't showing being hit since they are BODY searches
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
Matt, I have no desire to get into an argument or flaming contest with you. We agree that standard filters have a valuable place in this environment and we both use standard filters. We agree that neither of us have the desire to spend countless hours tweaking filters and that automated solutions are the way to simplify this effort. We have each taken different approaches using different methodologies and tools to do this, based on our own skill sets, backgrounds, need perceptions and other factors. We are both appreciative of the effort that many people have put into developing and maintaining these products and freely sharing them with us, and I'm sure that we're both willing to contribute in any way we can to assist in these efforts. We happen to disagree regarding the extent that these standard filters can be applied to our own specific environments. So be it. We also disagree on the value of analysis. So be it. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Robertson Sent: Monday, December 22, 2003 10:08 PM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. I understand all that stuff, George, but I disagree completely that you can't apply global, updated rules to some aspects of the problem. As such a global filter repository can make a huge dent in virtually everyone's workload. Do we really all need to create our own filters to remove p.en1s pi11z from our inbox? Is having the ability to more quickly react to new spam bad? Think of this as a virus definitiion list, except given Declude's modularity individuals can decide which virii they will allow themselves to be infected with. Nothing in this world is going to be perfect, and certainly you can write your own filters until you're blue in the face. I've been tinkering constantly with Declude for something like two years, and I expect to continue. But I also expect to automate as much of this -- or any other job -- as possible. I have more profitable and less aggravating things to do than this. I'm sure you do too. The community can benefit from some standardization and shared effort. Some here have already gone miles toward this goal, as many on this list know. I'm saying a Next Step should be taken, and anyone who wants to ignore the initiative is welcome to do so. --Matt-- --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.
Can we get a link to Kami's filters? Thanks.Neal MathewsNetwork Systems EngineerThe Carriage House Co.'s, Inc.[EMAIL PROTECTED] wrote: -To: [EMAIL PROTECTED]From: "Matt Robertson" [EMAIL PROTECTED]Sent by: [EMAIL PROTECTED]Date: 12/22/2003 06:13PMSubject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. I came into this thread late, so my comments may not be strictly on point, but it seems to me the solution to this is to only use filters that work. Duh, right? In other words let the community validate and update Filter X and you simply plug in what you please.That means a centralized filter storage, update and distribution site. We actually aren't so far off that mark now. Look at Kami Razvan's ftp site and you'll find a treasure trove of filters there. A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today.I implemented most of Kami's stuff last week (supplementing most of the filters already installed that came from Matt Bramble and the result is a massive surge in my attach-to-kill ratio (on the kill side). There are so many I had to aggressively reorganize my global.cfg, but the results have been splendid, with the most processor-intensive filters not kicking in unless needed.I wrote a ColdFusion routine that downloads my selected filters, alters them to suit my skip and max weights, and uploads them to my mail server (the filters are regularly updated). Anyone who wants a copy let me know.-Matt Robertson, [EMAIL PROTECTED]MSB Designs, Inc. http://mysecretbase.com[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]---This E-mail came from the Declude.JunkMail mailing list. Tounsubscribe, just send an E-mail to [EMAIL PROTECTED], andtype "unsubscribe Declude.JunkMail". The archives can be foundat http://www.mail-archive.com.--- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.
Hi; The filters are available for anyone who wishes to use it.. the challenge is to keep this link out of the hands of search engines. Imagine the keywords that our our company will be associated with. If you want to use the filters simply visit the ftp site: ftp://ftp.OUR DOMAIN/IMail OUR DOMAIN = ClickandPledge.com These filters are updated 4 times a day. There are also some timed filters. Like Blacklists- you can choose 30 days, 10 days, etc. the same goes with the URL in body filters. We soon will have the timed BlacklistinBody filters. Please make sure to read the README.txt file first. Regards, Kami From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED]Sent: Tuesday, December 23, 2003 8:27 AMTo: [EMAIL PROTECTED]Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. Can we get a link to Kami's filters? Thanks. Neal Mathews Network Systems Engineer The Carriage House Co.'s, Inc.[EMAIL PROTECTED] wrote: - To: [EMAIL PROTECTED]From: "Matt Robertson" [EMAIL PROTECTED]Sent by: [EMAIL PROTECTED]Date: 12/22/2003 06:13PMSubject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. I came into this thread late, so my comments may not be strictly on point, but it seems to me the solution to this is to only use filters that work. Duh, right? In other words let the community validate and update Filter X and you simply plug in what you please.That means a centralized filter storage, update and distribution site. We actually aren't so far off that mark now. Look at Kami Razvan's ftp site and you'll find a treasure trove of filters there. A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today.I implemented most of Kami's stuff last week (supplementing most of the filters already installed that came from Matt Bramble and the result is a massive surge in my attach-to-kill ratio (on the kill side). There are so many I had to aggressively reorganize my global.cfg, but the results have been splendid, with the most processor-intensive filters not kicking in unless needed.I wrote a ColdFusion routine that downloads my selected filters, alters them to suit my skip and max weights, and uploads them to my mail server (the filters are regularly updated). Anyone who wants a copy let me know.-Matt Robertson, [EMAIL PROTECTED]MSB Designs, Inc. http://mysecretbase.com[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]---This E-mail came from the Declude.JunkMail mailing list. Tounsubscribe, just send an E-mail to [EMAIL PROTECTED], andtype "unsubscribe Declude.JunkMail". The archives can be foundat http://www.mail-archive.com.--- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. functionality. functionality. functionality. functionality.
Thanks, Kami!Neal Mathews Network Systems Engineer The Carriage House Co.'s, Inc.[EMAIL PROTECTED] wrote: -To: [EMAIL PROTECTED]From: "Kami Razvan" [EMAIL PROTECTED]Sent by: [EMAIL PROTECTED]Date: 12/23/2003 09:16AMSubject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.Hi; The filters are available for anyone who wishes to use it.. the challenge is to keep this link out of the hands of search engines. Imagine the keywords that our our company will be associated with. If you want to use the filters simply visit the ftp site: ftp://ftp.OUR DOMAIN/IMail OUR DOMAIN = ClickandPledge.com These filters are updated 4 times a day. There are also some timed filters. Like Blacklists- you can choose 30 days, 10 days, etc. the same goes with the URL in body filters. We soon will have the timed BlacklistinBody filters. Please make sure to read the README.txt file first. Regards, Kami From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Tuesday, December 23, 2003 8:27 AM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. Can we get a link to Kami's filters? Thanks. Neal Mathews Network Systems Engineer The Carriage House Co.'s, Inc. [EMAIL PROTECTED] wrote: - To: [EMAIL PROTECTED] From: "Matt Robertson" [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] Date: 12/22/2003 06:13PM Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. I came into this thread late, so my comments may not be strictly on point, but it seems to me the solution to this is to only use filters that work. Duh, right? In other words let the community validate and update Filter X and you simply plug in what you please. That means a centralized filter storage, update and distribution site. We actually aren't so far off that mark now. Look at Kami Razvan's ftp site and you'll find a treasure trove of filters there. A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today. I implemented most of Kami's stuff last week (supplementing most of the filters already installed that came from Matt Bramble and the result is a massive surge in my attach-to-kill ratio (on the kill side). There are so many I had to aggressively reorganize my global.cfg, but the results have been splendid, with the most processor-intensive filters not kicking in unless needed. I wrote a ColdFusion routine that downloads my selected filters, alters them to suit my skip and max weights, and uploads them to my mail server (the filters are regularly updated). Anyone who wants a copy let me know. -- --- Matt Robertson, [EMAIL PROTECTED] MSB Designs, Inc. http://mysecretbase.com --- -- --- [This E-mail was scanned for viruses by Declude Virus ( http://www.declude.com )] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com . --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
George, Thanks again for the stats. These do verify that spammers are obfuscating the Yahoo redirection code and those lines need to stay in the filter as a result. At least I wasn't wasting my time when I came up with that stuff :) I didn't get too much else out of the results though. Maybe I'll reorder the test types in the OBFUSCATION filter, and I did make a change to what will become the next version of GIBBERISH where I moved the Words, Acronyms and Stock Market Symbols section below the Auto-generated Codes section, but I don't yet see any need to tweak the files line for line, only section by section because management is important. Matt George Kulman wrote: Matt, Here are two analyses. The 11-15 to 11-30 covers the period from when I implemented your filters until I began using SKIPIFWEIGHT and MAXWEIGHT which obviously has some effect on the stats. The 11-15 to 12-21 expands the prior set to include the additional filters. There's also the weighting effect to consider. While I run the OBFUSCATION and Y!DIRECTED at hold weight (15), I use the GIBBERISH like the COMMENTS test and accumulate weight per hit. Since my SKIPIFWEIGHT is set to my DELETE weight (60), the filters will run until that's reached. These stats aren't a big deal to produce since its all in a SQL database. I'll be implementing your new filter versions this coming weekend (with new names to avoid commingling stats). I do strip out comments since they become meaningless as the filter contents are resequenced by my system. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 10:32 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality. George, I think that logic can get you 95% of the way there with something as convoluted as this, that is run only about 1/3 of the time, and considering that you are only battling for about 2% of the processing power required by this filter alone, which shouldn't be too terribly much. Removing the comment blocks would probably have a bigger effect :) Changing to the new version of the filter should definitely help, though this isn't by far my most weighty filter. Here's something that I've very curious about though...the Y!DIRECTED filter contains a bunch of BODY searches for obfuscated strings, something that is almost totally redundant with the OBFUSCATION filter. I would be very curious to see how often those lines are hit because they could be dumped for a measurable performance increase. Any chance you want to take a crack at that? I wouldn't be surprised to see them never hit. Matt George Kulman wrote: Matt, I use LOGLEVEL HIGH for my data collection and analysis stuff and, as Bill pointed out, all hits are reflected. I've started to use SKIPIFWEIGHT. The result of course is that filters are bypassed and the statistics are skewed. For example on Friday 12/19, 15291 emails were processed by Declude on my system. Only 4604 were processed by the GIBBERISH filter. Of these 1328 had a total of 3854 hits. My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. If I collect all of the data, then I gain no benefit, since all of the processing takes place. If I take advantage of the analysis data, I reduce my processing workload but effectively destroy the validity of the statistical data which is now skewed by my filtering control. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 3:17 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. George, That's good data to have. I would have to assume that something tagged as gibberish in the main test would be random, and that's fairly well indicated by the somewhat tight range of the two character strings. Unless you are using a logging feature that I'm not aware of, you are only showing the last hit that the filter produces, and that explains why the Z strings are mostly bunched at the top. I've got these ordered alphabetically and will probably leave them there for management purposes. The counterbalances though are definitely something that I will use your information for reordering them. I believe I made an attempt to order these in the 2.0 filter version according to what I thought would be more common as well
Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
George, That's good data to have. I would have to assume that something tagged as gibberish in the main test would be random, and that's fairly well indicated by the somewhat tight range of the two character strings. Unless you are using a logging feature that I'm not aware of, you are only showing the last hit that the filter produces, and that explains why the Z strings are mostly bunched at the top. I've got these ordered alphabetically and will probably leave them there for management purposes. The counterbalances though are definitely something that I will use your information for reordering them. I believe I made an attempt to order these in the 2.0 filter version according to what I thought would be more common as well as what would be a faster search (BODY searches are slower than other things and will go lower in general, though a BODY search for base64 goes at the top because it is fairly common). Because of this and along with the above mentioned issue, the hit stats therefore aren't a perfect indication of what would save the most processing power, but it definitely helps if you just make some assumptions. I hadn't gathered any stats myself on the Auto-generated Codes that I added in about a month or so ago, and it's nice to see that they're getting hit since I was really just brainstorming about what types of things might be seen. I might remove some entries though if they aren't showing being hit since they are BODY searches and expensive. I'll probably still leave that list of Auto-generated Codes in alphabetical order though for management purposes. This shouldn't make a big difference considering that the most common one only gets hit about 1-3% of the time (don't know how common the filter fails a later line which ends up getting logged instead). If Declude did log every line that hits in a filter, you would see things like GIBBERISH hitting some attachments thousands of times per message, and I don't think that's worth the trouble. Data like this will make a much bigger impact on performance if you run it against filters where hits can only occur once in a file due to unique data or exact matching. Kami has a bunch of those. Thanks, Matt George Kulman wrote: Matt, I thought you might be interested in the attached data which analyzes the GIBBERISH and ANTI-GIBBERISH filters by number of hits on my system from 11/15 through yesterday. If you're looking for effectiveness you should set the entries in descending order of probability. I use a variation which looks at date of most recent hit as well as hit count, although that's more important with filters that are being modified on a continual rather that a fairly static filter such as these two. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 9:52 AM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. I've made some huge leaps forward recently in terms of the processing power required to run Declude with the custom filters that I have installed. This was done by way of the SKIPIFWEIGHT functionality introduced in the latest beta, but also by way of re-ordering my filters in the Global.cfg file so that the easiest to process custom filters are run first in the hopes of avoiding the need to run more costly ones. This new version of GIBBERISH makes use of functionality introduced in the 1.77 beta, however the most recent interim release, 1.77i7, should be used in order to guarantee proper operation (initial versions would always end processing, and effectively disabled the filters). The END functionality removes the need to have ANTI filters since the filter can be stopped before it gets to the main filter matches, and it also presents another opportunity to save on the processing power required to run such things. This also makes use of the MAXWEIGHT functionality to limit the max score as well as end processing once a single hit has been scored. Note that the filter will only log (at the LOW setting) and show WARN actions when the filter is tripped and an END was not hit...which is great! No more looking at non-scoring custom filters due to counterbalances :D Please read through the file and follow these instructions if you already have GIBBERISH installed: 1) Comment out the ANTI-GIBBERISH custom filter in your Global.cfg 2) Change the score of the GIBBERISH filter to 0 in your Global.cfg. 3) Change the scoring of the filter to match your system (it is scored by default for base 10 systems). This can be done by changing the MAXWEIGHT and Main Filter lines to reflect the multiple of 10 that your system is based on. 4) Change the SKIPIFWEIGHT score to reflect your delete weight, or whatever weight you would like for the filter to be skipped if the system has
Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
Matt, if you set your JunkMail logging to HIGH, you will see every line item that Declude matches on in the FILTER files Bill - Original Message - From: Matthew Bramble [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, December 22, 2003 12:17 PM Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. George, That's good data to have. I would have to assume that something tagged as gibberish in the main test would be random, and that's fairly well indicated by the somewhat tight range of the two character strings. Unless you are using a logging feature that I'm not aware of, you are only showing the last hit that the filter produces, and that explains why the Z strings are mostly bunched at the top. I've got these ordered alphabetically and will probably leave them there for management purposes. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.
Ick...but thanks for letting me know. Maybe this is better to have in debug. I could see some filters hitting even more than GIBBERISH does on Base64 stuff. Matt Bill Landry wrote: Matt, if you set your JunkMail logging to HIGH, you will see every line item that Declude matches on in the FILTER files Bill - Original Message - From: Matthew Bramble [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, December 22, 2003 12:17 PM Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. George, That's good data to have. I would have to assume that something tagged as gibberish in the main test would be random, and that's fairly well indicated by the somewhat tight range of the two character strings. Unless you are using a logging feature that I'm not aware of, you are only showing the last hit that the filter produces, and that explains why the Z strings are mostly bunched at the top. I've got these ordered alphabetically and will probably leave them there for management purposes. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- === Matthew S. Bramble President and Technical Coordinator iGaia Incorporated, Operator of NYcars.com --- Office Phone: (518) 862-9042 Cellular: (518) 229-3375 Fax: (518) 862-9044 E-mail: [EMAIL PROTECTED] or [EMAIL PROTECTED] === --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
Matt, I use LOGLEVEL HIGH for my data collection and analysis stuff and, as Bill pointed out, all hits are reflected. I've started to use SKIPIFWEIGHT. The result of course is that filters are bypassed and the statistics are skewed. For example on Friday 12/19, 15291 emails were processed by Declude on my system. Only 4604 were processed by the GIBBERISH filter. Of these 1328 had a total of 3854 hits. My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. If I collect all of the data, then I gain no benefit, since all of the processing takes place. If I take advantage of the analysis data, I reduce my processing workload but effectively destroy the validity of the statistical data which is now skewed by my filtering control. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 3:17 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. George, That's good data to have. I would have to assume that something tagged as gibberish in the main test would be random, and that's fairly well indicated by the somewhat tight range of the two character strings. Unless you are using a logging feature that I'm not aware of, you are only showing the last hit that the filter produces, and that explains why the Z strings are mostly bunched at the top. I've got these ordered alphabetically and will probably leave them there for management purposes. The counterbalances though are definitely something that I will use your information for reordering them. I believe I made an attempt to order these in the 2.0 filter version according to what I thought would be more common as well as what would be a faster search (BODY searches are slower than other things and will go lower in general, though a BODY search for base64 goes at the top because it is fairly common). Because of this and along with the above mentioned issue, the hit stats therefore aren't a perfect indication of what would save the most processing power, but it definitely helps if you just make some assumptions. I hadn't gathered any stats myself on the Auto-generated Codes that I added in about a month or so ago, and it's nice to see that they're getting hit since I was really just brainstorming about what types of things might be seen. I might remove some entries though if they aren't showing being hit since they are BODY searches and expensive. I'll probably still leave that list of Auto-generated Codes in alphabetical order though for management purposes. This shouldn't make a big difference considering that the most common one only gets hit about 1-3% of the time (don't know how common the filter fails a later line which ends up getting logged instead). If Declude did log every line that hits in a filter, you would see things like GIBBERISH hitting some attachments thousands of times per message, and I don't think that's worth the trouble. Data like this will make a much bigger impact on performance if you run it against filters where hits can only occur once in a file due to unique data or exact matching. Kami has a bunch of those. Thanks, Matt George Kulman wrote: Matt, I thought you might be interested in the attached data which analyzes the GIBBERISH and ANTI-GIBBERISH filters by number of hits on my system from 11/15 through yesterday. If you're looking for effectiveness you should set the entries in descending order of probability. I use a variation which looks at date of most recent hit as well as hit count, although that's more important with filters that are being modified on a continual rather that a fairly static filter such as these two. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 9:52 AM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. I've made some huge leaps forward recently in terms of the processing power required to run Declude with the custom filters that I have installed. This was done by way of the SKIPIFWEIGHT functionality introduced in the latest beta, but also by way of re-ordering my filters in the Global.cfg file so that the easiest to process custom filters are run first in the hopes of avoiding the need to run more costly ones. This new version of GIBBERISH makes use of functionality introduced in the 1.77 beta, however the most recent interim release, 1.77i7, should be used in order to guarantee proper
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. I came into this thread late, so my comments may not be strictly on point, but it seems to me the solution to this is to only use filters that work. Duh, right? In other words let the community validate and update Filter X and you simply plug in what you please. That means a centralized filter storage, update and distribution site. We actually aren't so far off that mark now. Look at Kami Razvan's ftp site and you'll find a treasure trove of filters there. A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today. I implemented most of Kami's stuff last week (supplementing most of the filters already installed that came from Matt Bramble and the result is a massive surge in my attach-to-kill ratio (on the kill side). There are so many I had to aggressively reorganize my global.cfg, but the results have been splendid, with the most processor-intensive filters not kicking in unless needed. I wrote a ColdFusion routine that downloads my selected filters, alters them to suit my skip and max weights, and uploads them to my mail server (the filters are regularly updated). Anyone who wants a copy let me know. -- --- Matt Robertson, [EMAIL PROTECTED] MSB Designs, Inc. http://mysecretbase.com --- -- --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
Matt, I do only use filters that work. There are a number of situations however that I believe make it impossible to effectively use only off the shelf filters. There are also valid reasons to perform my own analysis of filter effectiveness: First, everyone's spam mix is different, just as their e-mail mix is different. That's the first thing that Scott and others try to make clear to a newbie who's looking for a canned solution. Second, not everyone class the same things as spam. I have clients who use dating services and others who don't want that type of e-mail. What kind of complaints would you get if you implemented Ipswitch's URL list as is. I know that I'd have an FP rate that would hurt my effectiveness. I also provide secondary MX services for a number of clients and see a lot of spam attempting to back-door their mail servers. Third, I use many BODY and HEADER filters which range from a few lines to a few thousand lines. These consume a tremendous amount of processing overhead as Scott has pointed out, but I have found them to be the most effective at killing spam. They can be a pain to maintain without a database, ease of updating and dupe checking, automated filter file generation and analysis of effectiveness. Regarding analysis and sequencing of these filters and the use of SKIPIFWEIGHT and END in particular; if I can get 80% of the hits in the first 20% of the entries and eliminate the rest of the unneeded processing, I'd be pretty stupid not to. I was just bemoaning that I'd be giving up some data collection that's been a big help. Thanks to changes that Scott has made lately, at least at a LOGLEVEL HIGH, the ability to effectively use individual log lines for data collection have simplified and enhanced that process. Fourth, I like and use many single function filters, particularly Matt Bramble's and I thank him again for the time he has put into them and his generosity for sharing them freely. Every one of my clients has different needs and defines spam differently and the definitions, filters and actions have to reflect this. I, for one, will definitely pass on a central repository George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Robertson Sent: Monday, December 22, 2003 6:13 PM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. I came into this thread late, so my comments may not be strictly on point, but it seems to me the solution to this is to only use filters that work. Duh, right? In other words let the community validate and update Filter X and you simply plug in what you please. That means a centralized filter storage, update and distribution site. We actually aren't so far off that mark now. Look at Kami Razvan's ftp site and you'll find a treasure trove of filters there. A centralized filter repository would turn analysis of filter results into an academic exercise to satisfy curiosity, rather than the general necessity it is today. I implemented most of Kami's stuff last week (supplementing most of the filters already installed that came from Matt Bramble and the result is a massive surge in my attach-to-kill ratio (on the kill side). There are so many I had to aggressively reorganize my global.cfg, but the results have been splendid, with the most processor-intensive filters not kicking in unless needed. I wrote a ColdFusion routine that downloads my selected filters, alters them to suit my skip and max weights, and uploads them to my mail server (the filters are regularly updated). Anyone who wants a copy let me know. -- --- Matt Robertson, [EMAIL PROTECTED] MSB Designs, Inc. http://mysecretbase.com --- -- --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality.
I understand all that stuff, George, but I disagree completely that you can't apply global, updated rules to some aspects of the problem. As such a global filter repository can make a huge dent in virtually everyone's workload. Do we really all need to create our own filters to remove p.en1s pi11z from our inbox? Is having the ability to more quickly react to new spam bad? Think of this as a virus definitiion list, except given Declude's modularity individuals can decide which virii they will allow themselves to be infected with. Nothing in this world is going to be perfect, and certainly you can write your own filters until you're blue in the face. I've been tinkering constantly with Declude for something like two years, and I expect to continue. But I also expect to automate as much of this -- or any other job -- as possible. I have more profitable and less aggravating things to do than this. I'm sure you do too. The community can benefit from some standardization and shared effort. Some here have already gone miles toward this goal, as many on this list know. I'm saying a Next Step should be taken, and anyone who wants to ignore the initiative is welcome to do so. --Matt-- --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. functionality. functionality.
George, I think that logic can get you 95% of the way there with something as convoluted as this, that is run only about 1/3 of the time, and considering that you are only battling for about 2% of the processing power required by this filter alone, which shouldn't be too terribly much. Removing the comment blocks would probably have a bigger effect :) Changing to the new version of the filter should definitely help, though this isn't by far my most weighty filter. Here's something that I've very curious about though...the Y!DIRECTED filter contains a bunch of BODY searches for obfuscated strings, something that is almost totally redundant with the OBFUSCATION filter. I would be very curious to see how often those lines are hit because they could be dumped for a measurable performance increase. Any chance you want to take a crack at that? I wouldn't be surprised to see them never hit. Matt George Kulman wrote: Matt, I use LOGLEVEL HIGH for my data collection and analysis stuff and, as Bill pointed out, all hits are reflected. I've started to use SKIPIFWEIGHT. The result of course is that filters are bypassed and the statistics are skewed. For example on Friday 12/19, 15291 emails were processed by Declude on my system. Only 4604 were processed by the GIBBERISH filter. Of these 1328 had a total of 3854 hits. My quandary now is to decide whether to use the new control functions of SKIPIFWEIGHT, MAXWEIGHT and END to reduce processing overhead or to collect a full set of evaluation data by letting everything run. It's truly a catch-22 situation. If I collect all of the data, then I gain no benefit, since all of the processing takes place. If I take advantage of the analysis data, I reduce my processing workload but effectively destroy the validity of the statistical data which is now skewed by my filtering control. George -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Monday, December 22, 2003 3:17 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] GIBBERISH 2.0.1, single file filter with END functionality. functionality. George, That's good data to have. I would have to assume that something tagged as gibberish in the main test would be random, and that's fairly well indicated by the somewhat tight range of the two character strings. Unless you are using a logging feature that I'm not aware of, you are only showing the last hit that the filter produces, and that explains why the Z strings are mostly bunched at the top. I've got these ordered alphabetically and will probably leave them there for management purposes. The counterbalances though are definitely something that I will use your information for reordering them. I believe I made an attempt to order these in the 2.0 filter version according to what I thought would be more common as well as what would be a faster search (BODY searches are slower than other things and will go lower in general, though a BODY search for base64 goes at the top because it is fairly common). Because of this and along with the above mentioned issue, the hit stats therefore aren't a perfect indication of what would save the most processing power, but it definitely helps if you just make some assumptions. I hadn't gathered any stats myself on the Auto-generated Codes that I added in about a month or so ago, and it's nice to see that they're getting hit since I was really just brainstorming about what types of things might be seen. I might remove some entries though if they aren't showing being hit since they are BODY searches and expensive. I'll probably still leave that list of Auto-generated Codes in alphabetical order though for management purposes. This shouldn't make a big difference considering that the most common one only gets hit about 1-3% of the time (don't know how common the filter fails a later line which ends up getting logged instead). If Declude did log every line that hits in a filter, you would see things like GIBBERISH hitting some attachments thousands of times per message, and I don't think that's worth the trouble. Data like this will make a much bigger impact on performance if you run it against filters where hits can only occur once in a file due to unique data or exact matching. Kami has a bunch of those. Thanks, Matt George Kulman wrote: Matt, I thought you might be interested in the attached data which analyzes the GIBBERISH and ANTI-GIBBERISH filters by number of hits on my system from 11/15 through yesterday. If you're looking for effectiveness you should set the entries in descending order of probability. I use a variation which looks at date of most recent hit as well as hit count, although that's more important with filters that are being modified on a continual rather that a fairly static filter
