Re: [Declude.JunkMail] Clarification

[Matt]

I think that ordering the grouping would be great, but group all of the text based filters together and let their order of appearance dictate when they get run.  Right now custom filters do work like that, and I did work carefully with my ordering and SKIPIFWEIGHT settings in order to improve performance. The multi-test thing you mentioned though would need to work as an overlay to scoring because after every successive test, all of the multi-test actions would need to be checked in order for this to have an effect on processing, otherwise it would be best left to a filter file type, and run after the things that need to be executed. I'm not sure how much of an impact the RBL lookups have in terms of processing, but I'm thinking this is very small compared to the rest.  If Declude doesn't yet do this, performance here could be improved by only querying an RBL once regardless of the code that you are looking for, so for instance, only one lookup would cover all of the SORBS tests, and then the code returned could be applied to each test defined for that one address.  You would need to use the same domain though for all the SORBS tests instead of the different ones that they provide.  Declude might also already do this??? My list: 1) Ordering groups of tests. 2) Lumping all text filters into a group and allowing SKIPIFWEIGHT for all of them. 3) Improved efficiency on RBL's by combining lookups into one. I'm leaving off the external test SKIPIFWEIGHT thing because I think it might already be enabled by Declude.  The RBL's alone handle about 50% to 65% of my deletions, and DNS caching probably makes quick work of this stuff since the number of hosts is much smaller than the number of messages, especially for legitimate E-mail. Matt Todd Holt wrote: I can’t speak as to the internal organization of Junkmail.  It might be very difficult to separate the similar tests from each other.  Perhaps the similar tests could be grouped and the group could be placed in an order with other tests.   As to the tests we would like to see run first, certainly Sniffer!  More than 90% of our deleted messages fail the Sniffer test.  That is very indicative to us and I would like this one run first.  For us, if a message fails sniffer, it only needs to fail one or two other tests to be deleted.  It would be nice if we could perhaps run the internal Junkmail tests next (ie. BadHeaders, NoPostmaster, NoAbuse, Routing, HeloBogus, etc) because these don’t need to make any network calls to accomplish (ie. they only need the message itself).  Many of our deleted messages fail two or three of these and could be deleted immediately without further testing (assuming they failed Sniffer).  Next, run the tests that require only internal machine resources, such as fromfile and pattern matching tests.  Finally, and it sounds like all of the RBL tests are launched simultaneously, burn the bandwidth to test the RBLs, RevDNS, etc., that require network bandwidth.   This progressive approach would allow us to place the tests which we find most useful at the front of the test queue.  And have a weight at which testing is no longer considered useful in determining the fate of a message.   I think this approach could cover a feature request that I have seen here repeatedly: A way to combine test failures into a single outcome.  For example, a test called DefinitelyDelete defined as failed Sniffer and failed Spamcop and failed Badheaders.   Todd Holt Xidix Technologies, Inc Las Vegas, NV  USA www.xidix.com 702.319.4349   -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Matt Sent: Wednesday, January 21, 2004 1:36 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification   Todd, The RBL's and the built in Declude tests are very efficient.  The RBL's also go off all at once essentially, so there is no good way to stagger them one by one. I think it would be great though if Declude allowed us to set a SKIPIFWEIGHT value for FROMFILE's, IPFILE's, SPAMDOMAIN's and other associated text filters in a manner similar to the custom filters.  And also, allow us to set a SKIPIFWEIGHT value for external tests. It would be nice to be able to order some things, though I would assume that RBL's and then built-in technical tests would always be run in that order, but then which of the others would be run would be nice to be able to set.  My top request would be to stop external tests based on weight because I assume that this would be where the majority of overhead could lie on an otherwise well organized system.  I'm thinking this could be accomplished by Declude passing in a current score and having the individual external tests handle what they do on their own.  I'm thinking that this might already be possible though, but I'm not sure about what order they are processed in, and I'm not sure that among others, Sniffer handles such a thing currently. Matt Todd Holt wrote: The following has been suggested before and is similar: - Allow the Declude admin to set the order of test processing and stop processing if the weight reaches a specified limit.   Your concern at the time was: - If the admin places the tests in the wrong order it would be possible to exceed maximum allowed weight prior to running negative weight tests.   I think this is an education issue, just like everything else about administering software.   I also think this would reduce our processing time per message by allowing us to place the tests with the highest hit percentage first and stop processing early on most messages. (Over 90% of our delete weight messages failed Sniffer and either BadHeaders or SpamCop)  I would like to save the processing time if those push the weight into the delete range.  I could make sure to place the negative weight tests first.   Todd Holt Xidix Technologies, Inc Las Vegas, NV  USA www.xidix.com 702.319.4349         -----Original Message----- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED]] On Behalf Of R. Scott Perry Sent: Wednesday, January 21, 2004 8:42 AM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Clarification                 Is there a test, in the works, that will end all processing       of   any further filters.  Basically, exit all Declude processing, or is       it   best to use the SKIPWEIGHT, thanks,       There isn't anything like that in the works now, but it is something     that   we may end up adding.                                                       -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail     mailservers.   Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day     evaluation.   --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]   --- This E-mail came from the Declude.JunkMail mailing list.  To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail".  The archives can be found at http://www.mail-archive.com. --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)]         --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)]   --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]   --- This E-mail came from the Declude.JunkMail mailing list.  To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail".  The archives can be found at http://www.mail-archive.com.       -- ===================================================== MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ ===================================================== -- ===================================================== MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =====================================================