RE: [Declude.JunkMail] Test based on results of other tests
Folks, Is there a test that can be based on the results of 2 or more other specific tests? ex: an email that fails both HELOBOGUS and BADHEADERS would fail HELOHEAD and have x number of points added/deducted to it? Thanks Nick --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Test based on results of other tests
Is there a test that can be based on the results of 2 or more other specific tests? ex: an email that fails both HELOBOGUS and BADHEADERS would fail HELOHEAD and have x number of points added/deducted to it? No, that is not possible. It is something that has been requested, but it looks like a feature that few people would use. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you have been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Test based on results of other tests
Is there a test that can be based on the results of 2 or more other specific tests? ex: an email that fails both HELOBOGUS and BADHEADERS would fail HELOHEAD and have x number of points added/deducted to it? No, that is not possible. It is something that has been requested, but it looks like a feature that few people would use. Although I am not a programmer, the problem with having a test like that would require a redesign of declude.exe, so that the various junkmail tests are run in distinct separate sections. For example, it would have to say run all white tests, wait until finished, run all LP4R tests, wait until finished, run Filter tests, wait until finished, and so forth and then run iftestthentest last. That would slow the program down, and in an ISP scenario, that can cause problems when dealing with thousands of messages per hour. John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Test based on results of other tests
Scott, This feature would be of GREAT use. Many simply haven't thought out the implications of allowing the ability to combine tests. One example: the gentleman that wants to filter for specific names, but only one one domain -- this should allow setting that up. Adding the ability to combine results from two lists would make many of the tests much more effective (but, of course, more complicated to maintain). -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of R. Scott Perry Sent: Wednesday, September 03, 2003 1:35 PM To: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Test based on results of other tests Is there a test that can be based on the results of 2 or more other specific tests? ex: an email that fails both HELOBOGUS and BADHEADERS would fail HELOHEAD and have x number of points added/deducted to it? No, that is not possible. It is something that has been requested, but it looks like a feature that few people would use. -Scott --- Declude JunkMail: The advanced anti-spam solution for IMail mailservers. Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection. Find out what you have been missing: Ask for a free 30-day evaluation. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Test based on results of other tests
Actually, it could be a minor change to the processing -- at the $default$.junkmaillevel, rather than global.cfg -- as this is not a test, but a handling of the test results. It would mean order dependence, usually (or the processing of combining tests done first, then other handling done). The minor change being the ability to keep adding weight at this point in processing. Or, if no added weight were allowed, then a preprocessing of the $junkmail file could allow seeting pass/fail of combine tests, based on test results known at that point. The difficulty from a programming standpoint will depend on where it is implemented, what features are allowed (just failing a new test name or adding weight) and the modularity of the existing program code. As to slowing down the system -- you already have to wait until all tests and whitelist are processed for each message, before a final decision is made on the message. This should not make any difference there. -Original Message- From:John Tolmachoff Is there a test that can be based on the results of 2 or more other specific tests? ex: an email that fails both HELOBOGUS and BADHEADERS would fail HELOHEAD and have x number of points added/deducted to it? No, that is not possible. It is something that has been requested, but it looks like a feature that few people would use. Although I am not a programmer, the problem with having a test like that would require a redesign of declude.exe, so that the various junkmail tests are run in distinct separate sections. For example, it would have to say run all white tests, wait until finished, run all LP4R tests, wait until finished, run Filter tests, wait until finished, and so forth and then run iftestthentest last. That would slow the program down, and in an ISP scenario, that can cause problems when dealing with thousands of messages per hour. --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Test based on results of other tests
Actually, it could be a minor change to the processing -- at the $default$.junkmaillevel, rather than Global.cfg -- as this is not a test, but a handling of the test results. It would mean order dependence, usually (or the processing of combining tests done first, then other handling done). The minor change being the ability to keep adding weight at this point in processing. It is declude.exe itself that would have to be altered. You actually reminded me of how complex this would be. Both the Global.cfg and appropriate .junkmail file would have to be loaded into memory, some tests run, consult the files, other tests run, consult the files, final tests run, consult the files and so forth. Right now, declude.exe loads the Global.cfg to determine what tests to run, the after tests have been run, consults the appropriate .junkmail file to determine what action to take. John Tolmachoff MCSE CSSA Engineer/Consultant eServices For You www.eservicesforyou.com --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Test based on results of other tests
You actually reminded me of how complex this would be. Both the Global.cfg and appropriate .junkmail file would have to be loaded into memory, some tests run, consult the files, other tests run, consult the files, final tests run, consult the files and so forth. You are trying to make this much more difficult. Yes, declude would have to change --- as it does whenever any new test type is added. But there would be absolutely no need to run tests, consult files, etc. As far as loading the files into memory -- that already takes place (or declude would not work at all). Once loaded, in the part of the program that processes the $junkmail file (whichever one is relevant), a scan could be done for special lines (eg, TWOTESTS COMBINEAND TEST1 TEST2 or TWOTESTS COMBINEXOR TEST1 TEST2 -- since OR is not really necessary, but XOR and AND would be good logical tests). The new tests are added to the list of tests (already in memory) with pass/fail info. Then processing continues as usual. Really, not an extrememly large amount of work. No starting, stopping, etc. All tests would run as they do now -- no need to change that. Adding weights would be different and more flexible for some purposes, but just the above would be an extreme jump forward in setting up tests --- one example: if an email has certain words, we isolate it, as it MAY be porn (they are reviewed and deleted or requeued). There are some ip4r tests that identify possible sporn IP's -- we use these to add weight, but don't hold (due to FP's). But, if the email msg fails both, we would probably delete them outright and hold/review the rest. Certain mailing lists also tend to fail the suspect porn list due to either their subject (for instance, this list) or the users there -- but we would ignore them if we had the ability to combine the two pieces of info. Adding weights: simple here as well. Scan global.cfg - strip out combine tests, run all other tests as done now, in parallel or serial, not relevant. when all results are back, before ending the thread, process the combine tests and add weights to them as indicated.then pass control to the part of the program that handles .junkmail. Right now, declude.exe loads the Global.cfg to determine what tests to run, the after tests have been run, consults the appropriate .junkmail file to determine what action to take. Which is exactly what it would continue to do, if the combine tests were done in the $junkmail file. Not to mention, this gives ever more flexibility for combining tests. Karen (who considers this such an obvious solution to a programmer, but suspects the patent office would issue a patent for such a technique, based on no one else has filed one for it yet!). --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Test based on results of other tests
I'm with you on how this would be accomplished, though it would probably be a somewhat laborious rewrite in how scoring was handled in comparison to how it is handled now. Just guessing of course. This was actually my first feature request to Scott after purchasing the application some time ago, and it's about the 5th time I've seen it talked about in the past few weeks. While I have almost absolute faith in just a few blacklists (SpamCop for example), I would definitely combine many other blacklists that I have less faith in as one test...in other words if a piece of mail failed both FIVETEN-SPAM and SORBS-SPAM, then I would use the combined test to add on a hefty penalty for an automatic fail. I could do the same for two different open relay tests, figuring that if two know about it, then it is more likely being used for spam and more likely to be fixed by a responsible administrator rather than having their E-mail blocked over a longer term. I would probably also apply this multi-test penalty to things like NOABUSE and NOPOSTMASTER because I generally see just one failed unless it is spam and I score them both very low individually. I might even do something like credit points on two technical tests that are often failed together, SPAMHEADERS and HELOBOGUS for instance, and that would let me increase the scores on each individually (I'd have to research this one more before I could claim it would be effective). Maybe it will be a treat for v2.0 :) And speaking of patents, anyone ever hear of this one? http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1Sect2=HITOFFd=PALLp=1u=/netahtml/srchnum.htmr=1f=Gl=50s1='6,368,227'.WKU.OS=PN/6,368,227RS=PN/6,368,227 Matt Karen D. Oland wrote: You actually reminded me of how complex this would be. Both the Global.cfg and appropriate .junkmail file would have to be loaded into memory, some tests run, consult the files, other tests run, consult the files, final tests run, consult the files and so forth. You are trying to make this much more difficult. Yes, declude would have to change --- as it does whenever any new test type is added. But there would be absolutely no need to run tests, consult files, etc. As far as loading the files into memory -- that already takes place (or declude would not work at all). Once loaded, in the part of the program that processes the $junkmail file (whichever one is relevant), a scan could be done for special lines (eg, TWOTESTS COMBINEAND TEST1 TEST2 or TWOTESTS COMBINEXOR TEST1 TEST2 -- since OR is not really necessary, but XOR and AND would be good logical tests). The new tests are added to the list of tests (already in memory) with pass/fail info. Then processing continues as usual. Really, not an extrememly large amount of work. No starting, stopping, etc. All tests would run as they do now -- no need to change that. Adding weights would be different and more flexible for some purposes, but just the above would be an extreme jump forward in setting up tests --- one example: if an email has certain words, we isolate it, as it MAY be porn (they are reviewed and deleted or requeued). There are some ip4r tests that identify possible sporn IP's -- we use these to add weight, but don't hold (due to FP's). But, if the email msg fails both, we would probably delete them outright and hold/review the rest. Certain mailing lists also tend to fail the suspect porn list due to either their subject (for instance, this list) or the users there -- but we would ignore them if we had the ability to combine the two pieces of info. Adding weights: simple here as well. Scan global.cfg - strip out "combine" tests", run all other tests as done now, in parallel or serial, not relevant. when all results are back, before ending the thread, process the combine tests and add weights to them as indicated.then pass control to the part of the program that handles .junkmail. Right now, declude.exe loads the Global.cfg to determine what tests to run, the after tests have been run, consults the appropriate .junkmail file to determine what action to take. Which is exactly what it would continue to do, if the combine tests were done in the $junkmail file. Not to mention, this gives ever more flexibility for combining tests. Karen (who considers this such an obvious solution to a programmer, but suspects the patent office would issue a patent for such a technique, based on "no one else has filed one for it yet!).
Re: [Declude.JunkMail] Test based on results of other tests
Shoot, my link got munged. Here's what I was really talking about: Are patent methods patently absurd? http://news.com.com/2100-1023-962182.html " The patent office has granted patents for side-to-side swinging on a swing set and for making a peanut butter and jelly sandwich without a crust." Matt Matthew Bramble wrote: I'm with you on how this would be accomplished, though it would probably be a somewhat laborious rewrite in how scoring was handled in comparison to how it is handled now. Just guessing of course. This was actually my first feature request to Scott after purchasing the application some time ago, and it's about the 5th time I've seen it talked about in the past few weeks. While I have almost absolute faith in just a few blacklists (SpamCop for example), I would definitely combine many other blacklists that I have less faith in as one test...in other words if a piece of mail failed both FIVETEN-SPAM and SORBS-SPAM, then I would use the combined test to add on a hefty penalty for an automatic fail. I could do the same for two different open relay tests, figuring that if two know about it, then it is more likely being used for spam and more likely to be fixed by a responsible administrator rather than having their E-mail blocked over a longer term. I would probably also apply this multi-test penalty to things like NOABUSE and NOPOSTMASTER because I generally see just one failed unless it is spam and I score them both very low individually. I might even do something like credit points on two technical tests that are often failed together, SPAMHEADERS and HELOBOGUS for instance, and that would let me increase the scores on each individually (I'd have to research this one more before I could claim it would be effective). Maybe it will be a treat for v2.0 :) And speaking of patents, anyone ever hear of this one? http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1Sect2=HITOFFd=PALLp=1u=/netahtml/srchnum.htmr=1f=Gl=50s1='6,368,227'.WKU.OS=PN/6,368,227RS=PN/6,368,227 Matt Karen D. Oland wrote: You actually reminded me of how complex this would be. Both the Global.cfg and appropriate .junkmail file would have to be loaded into memory, some tests run, consult the files, other tests run, consult the files, final tests run, consult the files and so forth. You are trying to make this much more difficult. Yes, declude would have to change --- as it does whenever any new test type is added. But there would be absolutely no need to run tests, consult files, etc. As far as loading the files into memory -- that already takes place (or declude would not work at all). Once loaded, in the part of the program that processes the $junkmail file (whichever one is relevant), a scan could be done for special lines (eg, TWOTESTS COMBINEAND TEST1 TEST2 or TWOTESTS COMBINEXOR TEST1 TEST2 -- since OR is not really necessary, but XOR and AND would be good logical tests). The new tests are added to the list of tests (already in memory) with pass/fail info. Then processing continues as usual. Really, not an extrememly large amount of work. No starting, stopping, etc. All tests would run as they do now -- no need to change that. Adding weights would be different and more flexible for some purposes, but just the above would be an extreme jump forward in setting up tests --- one example: if an email has certain words, we isolate it, as it MAY be porn (they are reviewed and deleted or requeued). There are some ip4r tests that identify possible sporn IP's -- we use these to add weight, but don't hold (due to FP's). But, if the email msg fails both, we would probably delete them outright and hold/review the rest. Certain mailing lists also tend to fail the suspect porn list due to either their subject (for instance, this list) or the users there -- but we would ignore them if we had the ability to combine the two pieces of info. Adding weights: simple here as well. Scan global.cfg - strip out "combine" tests", run all other tests as done now, in parallel or serial, not relevant. when all results are back, before ending the thread, process the combine tests and add weights to them as indicated.then pass control to the part of the program that handles .junkmail. Right now, declude.exe loads the Global.cfg to determine what tests to run, the after tests have been run, consults the appropriate .junkmail file to determine what action to take. Which is exactly what it would continue to do, if the combine tests were done in the $junkmail file. Not to mention, this gives ever more flexibility for combining tests. Karen (who considers this such an obvious solution to a programmer, but suspects the patent office would issue a patent for such a technique, based on "no one else has filed one for it yet!).
Re: [Declude.JunkMail] Test based on results of other tests
Shoot, my link got munged. Here's what I was really talking about: Are patent methods patently absurd? http://news.com.com/2100-1023-962182.html " The patent office has granted patents for side-to-side swinging on a swing set and for making a peanut butter and jelly sandwich without a crust." Matt Matthew Bramble wrote: I'm with you on how this would be accomplished, though it would probably be a somewhat laborious rewrite in how scoring was handled in comparison to how it is handled now. Just guessing of course. This was actually my first feature request to Scott after purchasing the application some time ago, and it's about the 5th time I've seen it talked about in the past few weeks. While I have almost absolute faith in just a few blacklists (SpamCop for example), I would definitely combine many other blacklists that I have less faith in as one test...in other words if a piece of mail failed both FIVETEN-SPAM and SORBS-SPAM, then I would use the combined test to add on a hefty penalty for an automatic fail. I could do the same for two different open relay tests, figuring that if two know about it, then it is more likely being used for spam and more likely to be fixed by a responsible administrator rather than having their E-mail blocked over a longer term. I would probably also apply this multi-test penalty to things like NOABUSE and NOPOSTMASTER because I generally see just one failed unless it is spam and I score them both very low individually. I might even do something like credit points on two technical tests that are often failed together, SPAMHEADERS and HELOBOGUS for instance, and that would let me increase the scores on each individually (I'd have to research this one more before I could claim it would be effective). Maybe it will be a treat for v2.0 :) And speaking of patents, anyone ever hear of this one? http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1Sect2=HITOFFd=PALLp=1u=/netahtml/srchnum.htmr=1f=Gl=50s1='6,368,227'.WKU.OS=PN/6,368,227RS=PN/6,368,227 Matt Karen D. Oland wrote: You actually reminded me of how complex this would be. Both the Global.cfg and appropriate .junkmail file would have to be loaded into memory, some tests run, consult the files, other tests run, consult the files, final tests run, consult the files and so forth. You are trying to make this much more difficult. Yes, declude would have to change --- as it does whenever any new test type is added. But there would be absolutely no need to run tests, consult files, etc. As far as loading the files into memory -- that already takes place (or declude would not work at all). Once loaded, in the part of the program that processes the $junkmail file (whichever one is relevant), a scan could be done for special lines (eg, TWOTESTS COMBINEAND TEST1 TEST2 or TWOTESTS COMBINEXOR TEST1 TEST2 -- since OR is not really necessary, but XOR and AND would be good logical tests). The new tests are added to the list of tests (already in memory) with pass/fail info. Then processing continues as usual. Really, not an extrememly large amount of work. No starting, stopping, etc. All tests would run as they do now -- no need to change that. Adding weights would be different and more flexible for some purposes, but just the above would be an extreme jump forward in setting up tests --- one example: if an email has certain words, we isolate it, as it MAY be porn (they are reviewed and deleted or requeued). There are some ip4r tests that identify possible sporn IP's -- we use these to add weight, but don't hold (due to FP's). But, if the email msg fails both, we would probably delete them outright and hold/review the rest. Certain mailing lists also tend to fail the suspect porn list due to either their subject (for instance, this list) or the users there -- but we would ignore them if we had the ability to combine the two pieces of info. Adding weights: simple here as well. Scan global.cfg - strip out "combine" tests", run all other tests as done now, in parallel or serial, not relevant. when all results are back, before ending the thread, process the combine tests and add weights to them as indicated.then pass control to the part of the program that handles .junkmail. Right now, declude.exe loads the Global.cfg to determine what tests to run, the after tests have been run, consults the appropriate .junkmail file to determine what action to take. Which is exactly what it would continue to do, if the combine tests were done in the $junkmail file. Not to mention, this gives ever more flexibility for combining tests. Karen (who considers this such an obvious solution to a programmer, but suspects the patent office would issue a patent for such a technique, based on "no one else has filed one for it yet!). -- === Matthew S. Bramble President and
