Matt: The log file points idea would be a great help - simply show how much each filter contributed. I can't imagine this would be very hard to implement.. & would help a lot in figuring out the effectiveness of certain filters..
The idea of END has been discussed before and of course it is more prevalent to those of us that use a lot of filters. I have seen emails that have failed with a weight of 400 and we delete on 60. So the CPU could have taken care of other email instead of wasting time on the email that was deleted. We have seen a lot of help from the IMail delete action with IP4R tests. Deleting on 13 failed tests deletes a lot of spam before it even reaches Declude. Something like: End Weight>60 in the global statement could be an effective approach. Of course and END or RETURN statement in the filter would be great too... I think a lot of these ideas are great and will help the product.. I am just wondering how one can explain all these features to someone coming new onboard? I guess that is where the archives come into play.. Regards, Kami -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew Bramble Sent: Thursday, November 13, 2003 9:48 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] Request for additional filtering functionality Scott, As I continue to look for new potential in filtering, I have repeatedly come across some limitations which restrict what can be done effectively, difficulty in figuring the scoring of some variable filters, and challenges from the additional processing power required to counterbalance some filters, so I just wanted to request three different things which appear like they might be somewhat reasonable extensions to the current environment. I'm putting these all together in one message because, at least from my perspective, they are all related, and I didn't want to bother you repeatedly with such requests. Those requests are as follows: 1) Provide the score of a test in the logs, WARN function and %TESTSFAILED% variable. This would help with filters that have internal counterbalances or variable scoring so that an admin could quickly determine how many points were assessed. I would imagine that it could be turned on by way of one or three lines in the Global.cfg, i.e. SHOWPOINTS ON or SHOWLOGPOINTS ON SHOWWARNPOINTS ON SHOWTESTSFAILEDPOINTS ON When on, this would add the points scored to each of the types of entries as follows: LOG: 11/13/2003 20:43:02 Q331903a90080bcc8 Msg failed IPLINKED ([Score: 7] Message failed IPLINKED test (189)). Action=WARN. WARN: X-RBL-Warning: IPLINKED: [Score: 7] Message failed IPLINKED test (226) TESTSFAILED: X-Weight: 16 (REVDNS [0], IPNOTINMX [0], IPLINKED [7], SPAMCOP [9]) These changes would not only make scoring without custom filters much easier, it would definitely make more advanced configurations much easier to score and therefore make the system as a whole easier to administrate. 2) Provide a method of defeating a custom filter (zero points) based on failing a specially marked test. Having this capability would along with the above requested feature would remove the need to write convoluted systems to counterbalance custom filters with ANTI filters (or whatever you want to call them). So instead of having entries for allowed character strings, base64 encoding, certificates, etc., listed in both a GIBBERISH and ANTI-GIBBERISH filter for instance could be instead be listed in just one file making implementation much easier, more straightforward and saving on processing power that might be required to parse a fully redundant ANTI filter. You might even explore the possibility of using an END function so that tests listed at the top of a custom filter file meant to defeat the test will stop the rest of the filter from being processed and scored as 0 in the event that a trigger is matched. I might suggest a configuration like the following: BODY END CONTAINS base64 I believe that I could probably save between 10% and 30% of my processing power by having the ability to defeat a custom filter or at least not be required to use a combination of filters for counterbalancing. Custom filters with long lists of combinations are very expensive to process, and I have a feeling that my current dual 1 GHz server could only handle about 50,000 messages a day under the current configuration from what I am seeing in task manager (single Declude.exe processes reaching just over 50%). This is after I removed many of the extensive body filters that I was using for a short while. 3) Provide a method of defining a maximum and/or minimum number of points that a particular custom filter can score. This would allow for better use of filters that can produce multiple hits and are scored per hit. There have been a few occasions where I have attempted to code a filter where it increments the score for each hit for a line in a filter, but in order to protect from some instances of legitimate use of the character strings, it would be beneficial to limit the scoring to a more conservative range so that no matter how many hits were produced, a maximum or minimum score couldn't be exceeded. An example would be as follows, though I assume that this might be more appropriately configured in the Global.cfg file: MAXPOINTS 3 SUBJECT 1 CONTAINS a.a SUBJECT 1 CONTAINS a.b SUBJECT 1 CONTAINS a.c SUBJECT 1 CONTAINS a.d SUBJECT 1 CONTAINS a.e SUBJECT 1 CONTAINS a.f ... This change would open up more possibilities as it would limit FP's created by some methods. Thanks for the consideration. Matt --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
