Re: [Declude.JunkMail] Clarification
I think Todd pretty much hit the nail on the head. Remember, I proposed that we would put all negative weight filters first, then anything we considered mandatory (including particular tests and filters we want to log), then the QUITIFWEIGHT test would start. I do understand that it is desirable to run all tests if we want to measure the worth of each. However, I want to control when I test, and that is not 24/365 on everything. In production, I want a message to accumulate sufficient points to fail, and then stop testing it and go on to the next message. I delete at 30. Why would I care that a message accumulates 600 at the end of the run when it scores 35 on the first three tests and I could have tested other messages while this one accumulated the extra points? -Dave - Original Message - From: John Tolmachoff (Lists) [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, January 22, 2004 6:01 PM Subject: RE: [Declude.JunkMail] Clarification If a filter is skipped by SKIPIFWEIGHT, at that point I am not concerned about logging that filter, as I do not want it to run. Remember, SKIPIFWEIGHT is only for filters. However, what if a message gets a high weight early, but then would get a negative weight from a filter? You took action before the message had a chance to get the negative weight. What if you are checking to see the effectiveness of one test compared to others? If processing is stopped short, that test may not be run on all messages. John Tolmachoff Engineer/Consultant/Owner eServices For You -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Dave Doherty Sent: Thursday, January 22, 2004 1:00 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification John- Doesn't SKIPIFWEIGHT also defeat the logging of the skipped tests? -Dave Doherty Skywaves, Inc. - Original Message - From: John Tolmachoff (Lists) [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, January 22, 2004 1:04 PM Subject: RE: [Declude.JunkMail] Clarification I would like to see the SKIPIFWEIGHT option removed. If we had a conditional option to stop when a specific weight is reached, then there would be not need for SKIPIFWEIGHT. In addition, why would anyone use SKIPIFWEIGHT on less than every test...and why would anyone define one test with a different SKIPIFWEIGHT value than another test? This leads me back to a HOLDIFWEIGHT/DELETEIFWEIGHT logic which optionally stops processing when reached. Coming in late some my comments may be off. Scott has stated before that to stop all processing once a certain weight has been reached would be difficult and/or problematic. That is where SKIPIFWEIGHT comes in. I use SKIPIFWEIGHT on all body filters, as those are the most expensive in terms of CPU cost. I then have body filters listed in order, from most effective to least effective or specific target. Example, I have a custom body filter on my server for one client only. That is the last filter to run. Also, another reason to not stop processing is if you are doing log analysis and adjust filters or blocks based on that analysis. If you stop processing at say 35, but the message would have failed 5 other tests, those tests will then not show up in log analysis. John Tolmachoff Engineer/Consultant/Owner eServices For You --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Clarification
I would like to see the SKIPIFWEIGHT option removed. If we had a conditional option to stop when a specific weight is reached, then there would be not need for SKIPIFWEIGHT. In addition, why would anyone use SKIPIFWEIGHT on less than every test...and why would anyone define one test with a different SKIPIFWEIGHT value than another test? This leads me back to a HOLDIFWEIGHT/DELETEIFWEIGHT logic which optionally stops processing when reached. Coming in late some my comments may be off. Scott has stated before that to stop all processing once a certain weight has been reached would be difficult and/or problematic. That is where SKIPIFWEIGHT comes in. I use SKIPIFWEIGHT on all body filters, as those are the most expensive in terms of CPU cost. I then have body filters listed in order, from most effective to least effective or specific target. Example, I have a custom body filter on my server for one client only. That is the last filter to run. Also, another reason to not stop processing is if you are doing log analysis and adjust filters or blocks based on that analysis. If you stop processing at say 35, but the message would have failed 5 other tests, those tests will then not show up in log analysis. John Tolmachoff Engineer/Consultant/Owner eServices For You --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification
Todd, Initially I didn't understand why the complexity was necessary, however it really is in this case. We do gain by having the ability to set SKIPIFWEIGHT according to individual tests, for instance, in my negatively weighted PSEUDO-WHITE test, I set the SKIPIFWEIGHT higher than elsewhere just in case something gets clobbered by the RBL's and other tests. Also, you might want to skip over a very large negatively weighted test if a different threshold has already been reached. What the settings in individual files gives us is added flexibility at the cost of a little extra complexity. Regarding the other Global settings that you mentioned, keep in mind that these would only be useful on servers where everything is treated the same way, and you could only chose one level to stop processing on, not two, because after you stop, you can't keep going :) It might be nice though to have a SKIPIFLOWWEIGHT test that would stop processing if something scored under a certain number of points, this way a negatively weighted pseudo-white file or a combination of tests could be used to save on processing with the rest of the filters. Need for this seems somewhat limited at the moment, but it would provide benefit if done properly. SKIPIFWEIGHT could also just simply be appended with two number fields, one high, and one low, and Scott could make that backwards compatible I'm sure. Matt Todd Holt wrote: I would like to see the SKIPIFWEIGHT option removed. If we had a conditional option to stop when a specific weight is reached, then there would be not need for SKIPIFWEIGHT. In addition, why would anyone use SKIPIFWEIGHT on less than every test...and why would anyone define one test with a different SKIPIFWEIGHT value than another test? This leads me back to a HOLDIFWEIGHT/DELETEIFWEIGHT logic which optionally stops processing when reached. Relating to Dave's comments below: Would it not be more flexible to move the actionIFWEIGHT options to the .junkmail file to take advantage of the available scoping options (system/domain/user)? This is also more consistent with the existing .junkmail options such as HEADER, WARN, DELETE, HOLD... Todd Holt Xidix Technologies, Inc Las Vegas, NV USA www.xidix.com 702.319.4349 -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED]] On Behalf Of Dave Doherty Sent: Wednesday, January 21, 2004 7:29 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification Scott- I think this is a great idea. Once we know a message has passed the delete limit, why would we want to keep testing it in routine operations? Of course, we'd need to be able to turn it off when needed for debugging or whatever, but it would save a lot of processing time under normal conditions. My suggestion would be to define it in global.cfg (maybe QUITIFWEIGHT ?) and have it become active only when encountered in the junkmail file test sequence. That would let us group the positive tests first, then any tests we considered mandatory, then QUITIFWEIGHT would stop the processing at that point or any later point if the specified weight is met or exceeded. That would minimize the need for SKIPIFWEIGHT and other statements. My two cents worth, anyway. -Dave Doherty Skywaves, Inc. - Original Message - From: "R. Scott Perry" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, January 21, 2004 11:41 AM Subject: RE: [Declude.JunkMail] Clarification Is there a test, in the works, that will end all processing of any further filters. Basically, exit all Declude processing, or is it best to use the SKIPWEIGHT, thanks, There isn't anything like that in the works now, but it is something that we may end up adding. -Scott --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found
Re: [Declude.JunkMail] Clarification
John- Doesn't SKIPIFWEIGHT also defeat the logging of the skipped tests? -Dave Doherty Skywaves, Inc. - Original Message - From: John Tolmachoff (Lists) [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, January 22, 2004 1:04 PM Subject: RE: [Declude.JunkMail] Clarification I would like to see the SKIPIFWEIGHT option removed. If we had a conditional option to stop when a specific weight is reached, then there would be not need for SKIPIFWEIGHT. In addition, why would anyone use SKIPIFWEIGHT on less than every test...and why would anyone define one test with a different SKIPIFWEIGHT value than another test? This leads me back to a HOLDIFWEIGHT/DELETEIFWEIGHT logic which optionally stops processing when reached. Coming in late some my comments may be off. Scott has stated before that to stop all processing once a certain weight has been reached would be difficult and/or problematic. That is where SKIPIFWEIGHT comes in. I use SKIPIFWEIGHT on all body filters, as those are the most expensive in terms of CPU cost. I then have body filters listed in order, from most effective to least effective or specific target. Example, I have a custom body filter on my server for one client only. That is the last filter to run. Also, another reason to not stop processing is if you are doing log analysis and adjust filters or blocks based on that analysis. If you stop processing at say 35, but the message would have failed 5 other tests, those tests will then not show up in log analysis. John Tolmachoff Engineer/Consultant/Owner eServices For You --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Clarification
If a filter is skipped by SKIPIFWEIGHT, at that point I am not concerned about logging that filter, as I do not want it to run. Remember, SKIPIFWEIGHT is only for filters. However, what if a message gets a high weight early, but then would get a negative weight from a filter? You took action before the message had a chance to get the negative weight. What if you are checking to see the effectiveness of one test compared to others? If processing is stopped short, that test may not be run on all messages. John Tolmachoff Engineer/Consultant/Owner eServices For You -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Dave Doherty Sent: Thursday, January 22, 2004 1:00 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification John- Doesn't SKIPIFWEIGHT also defeat the logging of the skipped tests? -Dave Doherty Skywaves, Inc. - Original Message - From: John Tolmachoff (Lists) [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, January 22, 2004 1:04 PM Subject: RE: [Declude.JunkMail] Clarification I would like to see the SKIPIFWEIGHT option removed. If we had a conditional option to stop when a specific weight is reached, then there would be not need for SKIPIFWEIGHT. In addition, why would anyone use SKIPIFWEIGHT on less than every test...and why would anyone define one test with a different SKIPIFWEIGHT value than another test? This leads me back to a HOLDIFWEIGHT/DELETEIFWEIGHT logic which optionally stops processing when reached. Coming in late some my comments may be off. Scott has stated before that to stop all processing once a certain weight has been reached would be difficult and/or problematic. That is where SKIPIFWEIGHT comes in. I use SKIPIFWEIGHT on all body filters, as those are the most expensive in terms of CPU cost. I then have body filters listed in order, from most effective to least effective or specific target. Example, I have a custom body filter on my server for one client only. That is the last filter to run. Also, another reason to not stop processing is if you are doing log analysis and adjust filters or blocks based on that analysis. If you stop processing at say 35, but the message would have failed 5 other tests, those tests will then not show up in log analysis. John Tolmachoff Engineer/Consultant/Owner eServices For You --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Clarification
1. Place negative weight tests first. 2. While testing effectiveness of a single test, place it first or turn off the stop processing flag for a period of time. Todd Holt Xidix Technologies, Inc Las Vegas, NV USA www.xidix.com 702.319.4349 -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of John Tolmachoff (Lists) Sent: Thursday, January 22, 2004 3:01 PM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Clarification If a filter is skipped by SKIPIFWEIGHT, at that point I am not concerned about logging that filter, as I do not want it to run. Remember, SKIPIFWEIGHT is only for filters. However, what if a message gets a high weight early, but then would get a negative weight from a filter? You took action before the message had a chance to get the negative weight. What if you are checking to see the effectiveness of one test compared to others? If processing is stopped short, that test may not be run on all messages. John Tolmachoff Engineer/Consultant/Owner eServices For You -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Dave Doherty Sent: Thursday, January 22, 2004 1:00 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification John- Doesn't SKIPIFWEIGHT also defeat the logging of the skipped tests? -Dave Doherty Skywaves, Inc. - Original Message - From: John Tolmachoff (Lists) [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, January 22, 2004 1:04 PM Subject: RE: [Declude.JunkMail] Clarification I would like to see the SKIPIFWEIGHT option removed. If we had a conditional option to stop when a specific weight is reached, then there would be not need for SKIPIFWEIGHT. In addition, why would anyone use SKIPIFWEIGHT on less than every test...and why would anyone define one test with a different SKIPIFWEIGHT value than another test? This leads me back to a HOLDIFWEIGHT/DELETEIFWEIGHT logic which optionally stops processing when reached. Coming in late some my comments may be off. Scott has stated before that to stop all processing once a certain weight has been reached would be difficult and/or problematic. That is where SKIPIFWEIGHT comes in. I use SKIPIFWEIGHT on all body filters, as those are the most expensive in terms of CPU cost. I then have body filters listed in order, from most effective to least effective or specific target. Example, I have a custom body filter on my server for one client only. That is the last filter to run. Also, another reason to not stop processing is if you are doing log analysis and adjust filters or blocks based on that analysis. If you stop processing at say 35, but the message would have failed 5 other tests, those tests will then not show up in log analysis. John Tolmachoff Engineer/Consultant/Owner eServices For You --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Clarification
Scott, Is there a test, in the works, that will end all processing of any further filters. Basically, exit all Declude processing, or is it best to use the SKIPWEIGHT, thanks, Keith -Original Message- From: R. Scott Perry [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 21, 2004 10:44 AM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification That is all correct (just one note to clarify, though: END will stop processing that one filter, but other tests will run). -Scott At 10:35 AM 1/21/2004, Keith Johnson wrote: We are giving our Declude filters an overhaul this week, adding in all the functionality of the new 'beta' tests and I wanted to ensure I am good on the following: Using Examples SKIPWEIGHT 70 MAXWEIGHT 60 MINWEIGHT 20 MAXWEIGHT: If during the run of the filter the weight associated with the filter reaches 60 then that one filter exits. MINWEIGHT: If during the run of the filter the weight associated with the filter is at 20 or below then that one filter exits. SKIPWEIGHT: Upon start of the filter if the total weight prior to entering the filter is 70 then that one filter will not run END: If an END is reached anywhere in any filter, it will exit any further Declude processing will END Thanks for the aid and time. Keith --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Clarification
Is there a test, in the works, that will end all processing of any further filters. Basically, exit all Declude processing, or is it best to use the SKIPWEIGHT, thanks, There isn't anything like that in the works now, but it is something that we may end up adding. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Clarification
The following has been suggested before and is similar: - Allow the Declude admin to set the order of test processing and stop processing if the weight reaches a specified limit. Your concern at the time was: - If the admin places the tests in the wrong order it would be possible to exceed maximum allowed weight prior to running negative weight tests. I think this is an education issue, just like everything else about administering software. I also think this would reduce our processing time per message by allowing us to place the tests with the highest hit percentage first and stop processing early on most messages. (Over 90% of our delete weight messages failed Sniffer and either BadHeaders or SpamCop) I would like to save the processing time if those push the weight into the delete range. I could make sure to place the negative weight tests first. Todd Holt Xidix Technologies, Inc Las Vegas, NV USA www.xidix.com 702.319.4349 -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of R. Scott Perry Sent: Wednesday, January 21, 2004 8:42 AM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Clarification Is there a test, in the works, that will end all processing of any further filters. Basically, exit all Declude processing, or is it best to use the SKIPWEIGHT, thanks, There isn't anything like that in the works now, but it is something that we may end up adding. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification
Todd, The RBL's and the built in Declude tests are very efficient. The RBL's also go off all at once essentially, so there is no good way to stagger them one by one. I think it would be great though if Declude allowed us to set a SKIPIFWEIGHT value for FROMFILE's, IPFILE's, SPAMDOMAIN's and other associated text filters in a manner similar to the custom filters. And also, allow us to set a SKIPIFWEIGHT value for external tests. It would be nice to be able to order some things, though I would assume that RBL's and then built-in technical tests would always be run in that order, but then which of the others would be run would be nice to be able to set. My top request would be to stop external tests based on weight because I assume that this would be where the majority of overhead could lie on an otherwise well organized system. I'm thinking this could be accomplished by Declude passing in a current score and having the individual external tests handle what they do on their own. I'm thinking that this might already be possible though, but I'm not sure about what order they are processed in, and I'm not sure that among others, Sniffer handles such a thing currently. Matt Todd Holt wrote: The following has been suggested before and is similar: - Allow the Declude admin to set the order of test processing and stop processing if the weight reaches a specified limit. Your concern at the time was: - If the admin places the tests in the wrong order it would be possible to exceed maximum allowed weight prior to running negative weight tests. I think this is an education issue, just like everything else about administering software. I also think this would reduce our processing time per message by allowing us to place the tests with the highest hit percentage first and stop processing early on most messages. (Over 90% of our delete weight messages failed Sniffer and either BadHeaders or SpamCop) I would like to save the processing time if those push the weight into the delete range. I could make sure to place the negative weight tests first. Todd Holt Xidix Technologies, Inc Las Vegas, NV USA www.xidix.com 702.319.4349 -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED]] On Behalf Of R. Scott Perry Sent: Wednesday, January 21, 2004 8:42 AM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Clarification Is there a test, in the works, that will end all processing of any further filters. Basically, exit all Declude processing, or is it best to use the SKIPWEIGHT, thanks, There isn't anything like that in the works now, but it is something that we may end up adding. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail scanned for viruses by Declude Virus (http://www.declude.com)] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
RE: [Declude.JunkMail] Clarification
I cant speak as to the internal organization of Junkmail. It might be very difficult to separate the similar tests from each other. Perhaps the similar tests could be grouped and the group could be placed in an order with other tests. As to the tests we would like to see run first, certainly Sniffer! More than 90% of our deleted messages fail the Sniffer test. That is very indicative to us and I would like this one run first. For us, if a message fails sniffer, it only needs to fail one or two other tests to be deleted. It would be nice if we could perhaps run the internal Junkmail tests next (ie. BadHeaders, NoPostmaster, NoAbuse, Routing, HeloBogus, etc) because these dont need to make any network calls to accomplish (ie. they only need the message itself). Many of our deleted messages fail two or three of these and could be deleted immediately without further testing (assuming they failed Sniffer). Next, run the tests that require only internal machine resources, such as fromfile and pattern matching tests. Finally, and it sounds like all of the RBL tests are launched simultaneously, burn the bandwidth to test the RBLs, RevDNS, etc., that require network bandwidth. This progressive approach would allow us to place the tests which we find most useful at the front of the test queue. And have a weight at which testing is no longer considered useful in determining the fate of a message. I think this approach could cover a feature request that I have seen here repeatedly: A way to combine test failures into a single outcome. For example, a test called DefinitelyDelete defined as failed Sniffer and failed Spamcop and failed Badheaders. Todd Holt Xidix Technologies, Inc Las Vegas, NV USA www.xidix.com 702.319.4349 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Wednesday, January 21, 2004 1:36 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification Todd, The RBL's and the built in Declude tests are very efficient. The RBL's also go off all at once essentially, so there is no good way to stagger them one by one. I think it would be great though if Declude allowed us to set a SKIPIFWEIGHT value for FROMFILE's, IPFILE's, SPAMDOMAIN's and other associated text filters in a manner similar to the custom filters. And also, allow us to set a SKIPIFWEIGHT value for external tests. It would be nice to be able to order some things, though I would assume that RBL's and then built-in technical tests would always be run in that order, but then which of the others would be run would be nice to be able to set. My top request would be to stop external tests based on weight because I assume that this would be where the majority of overhead could lie on an otherwise well organized system. I'm thinking this could be accomplished by Declude passing in a current score and having the individual external tests handle what they do on their own. I'm thinking that this might already be possible though, but I'm not sure about what order they are processed in, and I'm not sure that among others, Sniffer handles such a thing currently. Matt Todd Holt wrote: The following has been suggested before and is similar:- Allow the Declude admin to set the order of test processing and stopprocessing if the weight reaches a specified limit.Your concern at the time was:- If the admin places the tests in the wrong order it would be possibleto exceed maximum allowed weight prior to running negative weight tests.I think this is an education issue, just like everything else aboutadministering software.I also think this would reduce our processing time per message byallowing us to place the tests with the highest hit percentage first andstop processing early on most messages. (Over 90% of our delete weightmessages failed Sniffer and either BadHeaders or SpamCop) I would liketo save the processing time if those push the weight into the deleterange. I could make sure to place the negative weight tests first.Todd HoltXidix Technologies, IncLas Vegas, NV USAwww.xidix.com702.319.4349 -Original Message-From: [EMAIL PROTECTED] [mailto:Declude.JunkMail-[EMAIL PROTECTED]] On Behalf Of R. Scott PerrySent: Wednesday, January 21, 2004 8:42 AMTo: [EMAIL PROTECTED]Subject: RE: [Declude.JunkMail] Clarification Is there a test, in the works, that will end all processing of any further filters. Basically, exit all Declude processing, or is it best to use the SKIPWEIGHT, thanks, There isn't anything like that in the works now, but it is something that we may end up adding. -Scott---Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailservervulnerability detection.Find out what you've been missing: Ask about our free 30-day evaluation. ---[This E-mail was scanned for viruses by Declude Virus(http://www.declude.com)]---This E-mail came
Re: [Declude.JunkMail] Clarification
I'm not sure how much of an impact the RBL lookups have in terms of processing, but I'm thinking this is very small compared to the rest. If Declude doesn't yet do this, performance here could be improved by only querying an RBL once regardless of the code that you are looking for, so for instance, only one lookup would cover all of the SORBS tests, and then the code returned could be applied to each test defined for that one address. You would need to use the same domain though for all the SORBS tests instead of the different ones that they provide. Declude might also already do this??? Yes, Declude already does this. Early versions would run the tests separately, but for quite some time now if there are 2 DNS-based spam tests that use the same zone (such as SORBS-SPAM and SORBS-DUHL), only one query will be made. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification
I think that ordering the grouping would be great, but group all of the text based filters together and let their order of appearance dictate when they get run. Right now custom filters do work like that, and I did work carefully with my ordering and SKIPIFWEIGHT settings in order to improve performance. The multi-test thing you mentioned though would need to work as an overlay to scoring because after every successive test, all of the multi-test actions would need to be checked in order for this to have an effect on processing, otherwise it would be best left to a filter file type, and run after the things that need to be executed. I'm not sure how much of an impact the RBL lookups have in terms of processing, but I'm thinking this is very small compared to the rest. If Declude doesn't yet do this, performance here could be improved by only querying an RBL once regardless of the code that you are looking for, so for instance, only one lookup would cover all of the SORBS tests, and then the code returned could be applied to each test defined for that one address. You would need to use the same domain though for all the SORBS tests instead of the different ones that they provide. Declude might also already do this??? My list: 1) Ordering groups of tests. 2) Lumping all text filters into a group and allowing SKIPIFWEIGHT for all of them. 3) Improved efficiency on RBL's by combining lookups into one. I'm leaving off the external test SKIPIFWEIGHT thing because I think it might already be enabled by Declude. The RBL's alone handle about 50% to 65% of my deletions, and DNS caching probably makes quick work of this stuff since the number of hosts is much smaller than the number of messages, especially for legitimate E-mail. Matt Todd Holt wrote: I cant speak as to the internal organization of Junkmail. It might be very difficult to separate the similar tests from each other. Perhaps the similar tests could be grouped and the group could be placed in an order with other tests. As to the tests we would like to see run first, certainly Sniffer! More than 90% of our deleted messages fail the Sniffer test. That is very indicative to us and I would like this one run first. For us, if a message fails sniffer, it only needs to fail one or two other tests to be deleted. It would be nice if we could perhaps run the internal Junkmail tests next (ie. BadHeaders, NoPostmaster, NoAbuse, Routing, HeloBogus, etc) because these dont need to make any network calls to accomplish (ie. they only need the message itself). Many of our deleted messages fail two or three of these and could be deleted immediately without further testing (assuming they failed Sniffer). Next, run the tests that require only internal machine resources, such as fromfile and pattern matching tests. Finally, and it sounds like all of the RBL tests are launched simultaneously, burn the bandwidth to test the RBLs, RevDNS, etc., that require network bandwidth. This progressive approach would allow us to place the tests which we find most useful at the front of the test queue. And have a weight at which testing is no longer considered useful in determining the fate of a message. I think this approach could cover a feature request that I have seen here repeatedly: A way to combine test failures into a single outcome. For example, a test called DefinitelyDelete defined as failed Sniffer and failed Spamcop and failed Badheaders. Todd Holt Xidix Technologies, Inc Las Vegas, NV USA www.xidix.com 702.319.4349 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Matt Sent: Wednesday, January 21, 2004 1:36 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification Todd, The RBL's and the built in Declude tests are very efficient. The RBL's also go off all at once essentially, so there is no good way to stagger them one by one. I think it would be great though if Declude allowed us to set a SKIPIFWEIGHT value for FROMFILE's, IPFILE's, SPAMDOMAIN's and other associated text filters in a manner similar to the custom filters. And also, allow us to set a SKIPIFWEIGHT value for external tests. It would be nice to be able to order some things, though I would assume that RBL's and then built-in technical tests would always be run in that order, but then which of the others would be run would be nice to be able to set. My top request would be to stop external tests based on weight because I assume that this would be where the majority of overhead could lie on an otherwise well organized system. I'm thinking this could be accomplished by Declude passing in a current score and having the individual external tests handle what they do on their own. I'm thinking that this might already be possible though, but I'm not sure about what order they are processed in, and I'm not sure that among others, Sniffer handles such a thing
Re: [Declude.JunkMail] Clarification
Scott- I think this is a great idea. Once we know a message has passed the delete limit, why would we want to keep testing it in routine operations? Of course, we'd need to be able to turn it off when needed for debugging or whatever, but it would save a lot of processing time under normal conditions. My suggestion would be to define it in global.cfg (maybe QUITIFWEIGHT ?) and have it become active only when encountered in the junkmail file test sequence. That would let us group the positive tests first, then any tests we considered mandatory, then QUITIFWEIGHT would stop the processing at that point or any later point if the specified weight is met or exceeded. That would minimize the need for SKIPIFWEIGHT and other statements. My two cents worth, anyway. -Dave Doherty Skywaves, Inc. - Original Message - From: R. Scott Perry [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, January 21, 2004 11:41 AM Subject: RE: [Declude.JunkMail] Clarification Is there a test, in the works, that will end all processing of any further filters. Basically, exit all Declude processing, or is it best to use the SKIPWEIGHT, thanks, There isn't anything like that in the works now, but it is something that we may end up adding. -Scott --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification..needed
Scott, Sure seems to work like a charm. Again - very kool! -Nick There is a new interim release (1.76i28) at http://www.declude.com/release/176i/declude.exe that changes the way that the weight is calculated (in i27 it would count negative weights, but no longer will), and adds logging at LOGLEVEL HIGH that should help determine if there are other issues. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification..needed
- Original Message - From: R. Scott Perry [EMAIL PROTECTED] There is a new interim release (1.76i28) at http://www.declude.com/release/176i/declude.exe that changes the way that the weight is calculated (in i27 it would count negative weights, but no longer will), and adds logging at LOGLEVEL HIGH that should help determine if there are other issues. Just for a bit of further clarification, when you say it will no longer count negative weights, does that mean the score will not be reduced by negative weights, even if they are list first in a filter file? Bill --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification..needed
There is a new interim release (1.76i28) at http://www.declude.com/release/176i/declude.exe that changes the way that the weight is calculated (in i27 it would count negative weights, but no longer will), and adds logging at LOGLEVEL HIGH that should help determine if there are other issues. Just for a bit of further clarification, when you say it will no longer count negative weights, does that mean the score will not be reduced by negative weights, even if they are list first in a filter file? That just refers to weights in the last column in the test definitions. For example, the IPNOTINMX test ends in ... 0 -3. That -3 was originally used in counting the weight (if the E-mail did not or had not yet failed the test), but no longer is. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification..needed
- Original Message - From: R. Scott Perry [EMAIL PROTECTED] There is a new interim release (1.76i28) at http://www.declude.com/release/176i/declude.exe that changes the way that the weight is calculated (in i27 it would count negative weights, but no longer will), and adds logging at LOGLEVEL HIGH that should help determine if there are other issues. Just for a bit of further clarification, when you say it will no longer count negative weights, does that mean the score will not be reduced by negative weights, even if they are list first in a filter file? That just refers to weights in the last column in the test definitions. For example, the IPNOTINMX test ends in ... 0 -3. That -3 was originally used in counting the weight (if the E-mail did not or had not yet failed the test), but no longer is. I'm sorry, Scott, I must have a mental block here. Are you saying that the -3 for the IPNOTINMX test is not deducted before the filter tests are run, but would still be deducted after all of the filter tests have been run, and before final disposition of the message? Sorry for being dense, I just want to make sure I understand the new scoring process. Thanks, Bill --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification..needed
That just refers to weights in the last column in the test definitions. For example, the IPNOTINMX test ends in ... 0 -3. That -3 was originally used in counting the weight (if the E-mail did not or had not yet failed the test), but no longer is. I'm sorry, Scott, I must have a mental block here. Are you saying that the -3 for the IPNOTINMX test is not deducted before the filter tests are run, but would still be deducted after all of the filter tests have been run, and before final disposition of the message? Sorry for being dense, I just want to make sure I understand the new scoring process. The weight of the E-mail will remain the same. All that changes is the way that the SKIPIFWEIGHT option calculates the weight to determine whether or not to skip the test. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Clarification..needed
I added the following: SKIPIFWEIGHT 60 MAXWEIGHT 60 I thought this would force the filter to skip if the weight is 60 and if the weight inside the filter reaches 60 it should also exit. Interestingly enough none of the filters that have this at the top ever execute. There is a new interim release (1.76i28) at http://www.declude.com/release/176i/declude.exe that changes the way that the weight is calculated (in i27 it would count negative weights, but no longer will), and adds logging at LOGLEVEL HIGH that should help determine if there are other issues. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Clarification..needed
Hi Scott: What I did was to move all of our negative weights to the top of the global.cfg file. None of our filter files have mixed positive negative weights- they are either all positive or all negative. Considering what you stated earlier that the tests are run according to the order they are listed in the global.cfg it seems like moving the negative weight tests to the top is the right thing to do. After adding the two lines to the top (considering we delete on 60) and again considering all the negative weights are done first the filters that have the two lines were expected not to run only if the weight is 60. The following is the order I had things listed. 1: IP4r, spamdomains, helobogus... Etc. 2: Negative tests 3: Filters with the two lines at top. Our spam folder all of a sudden had about 20 emails in it all at weights of 30 -50.. And none have any of the filters listed. The log file shows none of the filters being run. I commented the two lines and will do more test tomorrow.. Regards, Kami -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of R. Scott Perry Sent: Tuesday, November 25, 2003 6:53 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Clarification..needed I added the following: SKIPIFWEIGHT 60 MAXWEIGHT 60 I thought this would force the filter to skip if the weight is 60 and if the weight inside the filter reaches 60 it should also exit. Interestingly enough none of the filters that have this at the top ever execute. There is a new interim release (1.76i28) at http://www.declude.com/release/176i/declude.exe that changes the way that the weight is calculated (in i27 it would count negative weights, but no longer will), and adds logging at LOGLEVEL HIGH that should help determine if there are other issues. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you've been missing: Ask about our free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
