Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
Hi, Markus, Thanks for responding. Well I went ahead and did it. I've rescaled everything to have 100 points be my HOLD weight. It was pretty easy because my previous HOLD weight was 5 so I just had to multiply everything by 20 to keep thingsrelative. Now, that I have it there I would like to re-tune some of my weights. In your system, if you have a test like HELOBOGUS, for example, how do you decide what weight to give HELOBOGUS? I was thinking that if I had the correct statistics about which types of messages, spam or legit, were flagged by which tests it would be pretty straightforward. For example, if I knew that of 1000 messages that were flagged as failing the HELOBOGUS test and 37% of them were legit messages and 73% setting the weight of HELOBOGUS to 73 would be statistically sound. Is my thinking correct on that or am I way off base? Thanks, Dan Geiser [EMAIL PROTECTED] - Original Message - From: Markus Gufler To: [EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:25 PM Subject: RE: [Declude.JunkMail] Scaling Up The Declude Weighting System Dan, We've choosen to scale up the weighting system exactly for the two reasons you've mentioned below: -more granularity -absolute weight and percentage is the same Note that there are some good filter files maintained by other Declude users that are updated regulary and has the "inside" weights set up for a Hold-on-20 weighting system. Markus From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan GeiserSent: Tuesday, April 20, 2004 4:48 PMTo: [EMAIL PROTECTED]Subject: [Declude.JunkMail] Scaling Up The Declude Weighting System Hello, All, Over the year or so that I've been reading the discussions on this list it seems I've read quite a bit about people scaling their weights up, i.e. instead of having a HOLD weight of 10, you might have a HOLD weight of 100 and then you adjust the corresponding test weights accordingly. Assuming that what I've read is correct, for those who uses this scaled up system... What sort of benefit is is that you feel that you receive from doing this? Does it allow a more granular tuning of your weighting system? Are there any other benefits I'm not thinking of? Does having a hold weight of 100, for example, help you think more clearly about each test being a percentage of the overall HOLD weight? I'm doing a major overhaul of Declude JunkMail configuration and I figured if a scaled up weight system is the best way to do things then I might want to implement that now. Thanks In Advance For Your Comments! Dan Geiser [EMAIL PROTECTED]
Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
If a test false positived 37% of the time, I certainly wouldn't be weighing it that high. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 12:57PM Hi, Markus, Thanks for responding. Well I went ahead and did it. I've rescaled everything to have 100 points be my HOLD weight. It was pretty easy because my previous HOLD weight was 5 so I just had to multiply everything by 20 to keep things relative. Now, that I have it there I would like to re-tune some of my weights. In your system, if you have a test like HELOBOGUS, for example, how do you decide what weight to give HELOBOGUS? I was thinking that if I had the correct statistics about which types of messages, spam or legit, were flagged by which tests it would be pretty straightforward. For example, if I knew that of 1000 messages that were flagged as failing the HELOBOGUS test and 37% of them were legit messages and 73% setting the weight of HELOBOGUS to 73 would be statistically sound. Is my thinking correct on that or am I way off base? Thanks, Dan Geiser [EMAIL PROTECTED] - Original Message - From: Markus Gufler To: [EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:25 PM Subject: RE: [Declude.JunkMail] Scaling Up The Declude Weighting System Dan, We've choosen to scale up the weighting system exactly for the two reasons you've mentioned below: -more granularity -absolute weight and percentage is the same Note that there are some good filter files maintained by other Declude users that are updated regulary and has the inside weights set up for a Hold-on-20 weighting system. Markus From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Geiser Sent: Tuesday, April 20, 2004 4:48 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] Scaling Up The Declude Weighting System Hello, All, Over the year or so that I've been reading the discussions on this list it seems I've read quite a bit about people scaling their weights up, i.e. instead of having a HOLD weight of 10, you might have a HOLD weight of 100 and then you adjust the corresponding test weights accordingly. Assuming that what I've read is correct, for those who uses this scaled up system... What sort of benefit is is that you feel that you receive from doing this? Does it allow a more granular tuning of your weighting system? Are there any other benefits I'm not thinking of? Does having a hold weight of 100, for example, help you think more clearly about each test being a percentage of the overall HOLD weight? I'm doing a major overhaul of Declude JunkMail configuration and I figured if a scaled up weight system is the best way to do things then I might want to implement that now. Thanks In Advance For Your Comments! Dan Geiser [EMAIL PROTECTED] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
I did exactly this when we added SPAMCHK as a test last year. I believe they recommended this range because spamchk would add a lot of small weights and a 1-10 scale is too narrow. It also allows us to create filters with words that are more common in non-spam, but more likely to be spam in higher frequency. That is, a dozen or so words that have a weight of 2 or 3 out of 100 would give me the desired final weight. But the best I could do on a 1-10 scale is give each 1 point which would put me over my hold weight pretty quick. --Todd. Dan Geiser wrote: Hello, All, Over the year or so that I've been reading the discussions on this list it seems I've read quite a bit about people scaling their weights up, i.e. instead of having a HOLD weight of 10, you might have a HOLD weight of 100 and then you adjust the corresponding test weights accordingly. Assuming that what I've read is correct, for those who uses this scaled up system... What sort of benefit is is that you feel that you receive from doing this? Does it allow a more granular tuning of your weighting system? Are there any other benefits I'm not thinking of? Does having a hold weight of 100, for example, help you think more clearly about each test being a percentage of the overall HOLD weight? I'm doing a major overhaul of Declude JunkMail configuration and I figured if a scaled up weight system is the best way to do things then I might want to implement that now. Thanks In Advance For Your Comments! Dan Geiser [EMAIL PROTECTED]
Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
Hi, Scott, Thanks for the feedback. The more I thought about it after sending the e-mail a few minutes ago the more certain I was that my logic was not. in fact not even remotely close to being sound. It really has to be thought of as a factor of multiple tests and not just one, so I understand what you are saying. But I have to disagree with your terminology. I wasn't describing a false positive situation. I don't think the HELOBOGUS test by itself can have a false positive. A message either passes or fails the HELOBOGUS test. If a message fails the HELOBOGUS test, meaning the HELO is bogus by Scott's criterion, yet that message is not a spam message, i.e. it is a legit e-mail, it doesn't mean that the HELOBOGUS generated a false positive. The HELO either truly is BOGUS or NOT BOGUS. If HELOBOGUS misidentified a message as being BOGUS that was NOT BOGUS then, yes, I think that would be a false positive. But by it's nature one single test cannot create a false positive unless the program code for the test is written incorrectly. Just my thoughts. Dan - Original Message - From: Scott Fisher [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:20 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System If a test false positived 37% of the time, I certainly wouldn't be weighing it that high. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 12:57PM Hi, Markus, Thanks for responding. Well I went ahead and did it. I've rescaled everything to have 100 points be my HOLD weight. It was pretty easy because my previous HOLD weight was 5 so I just had to multiply everything by 20 to keep things relative. Now, that I have it there I would like to re-tune some of my weights. In your system, if you have a test like HELOBOGUS, for example, how do you decide what weight to give HELOBOGUS? I was thinking that if I had the correct statistics about which types of messages, spam or legit, were flagged by which tests it would be pretty straightforward. For example, if I knew that of 1000 messages that were flagged as failing the HELOBOGUS test and 37% of them were legit messages and 73% setting the weight of HELOBOGUS to 73 would be statistically sound. Is my thinking correct on that or am I way off base? Thanks, Dan Geiser [EMAIL PROTECTED] - Original Message - From: Markus Gufler To: [EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:25 PM Subject: RE: [Declude.JunkMail] Scaling Up The Declude Weighting System Dan, We've choosen to scale up the weighting system exactly for the two reasons you've mentioned below: -more granularity -absolute weight and percentage is the same Note that there are some good filter files maintained by other Declude users that are updated regulary and has the inside weights set up for a Hold-on-20 weighting system. Markus From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Geiser Sent: Tuesday, April 20, 2004 4:48 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] Scaling Up The Declude Weighting System Hello, All, Over the year or so that I've been reading the discussions on this list it seems I've read quite a bit about people scaling their weights up, i.e. instead of having a HOLD weight of 10, you might have a HOLD weight of 100 and then you adjust the corresponding test weights accordingly. Assuming that what I've read is correct, for those who uses this scaled up system... What sort of benefit is is that you feel that you receive from doing this? Does it allow a more granular tuning of your weighting system? Are there any other benefits I'm not thinking of? Does having a hold weight of 100, for example, help you think more clearly about each test being a percentage of the overall HOLD weight? I'm doing a major overhaul of Declude JunkMail configuration and I figured if a scaled up weight system is the best way to do things then I might want to implement that now. Thanks In Advance For Your Comments! Dan Geiser [EMAIL PROTECTED] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- Sign up for virus-free and spam-free e-mail with Nexus Technology Group http://www.nexustechgroup.com/mailscan --- Sign up for virus-free and spam-free e-mail with Nexus Technology Group http://www.nexustechgroup.com/mailscan --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from
Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
Dan, Individual tests do not false positive (unless they are poorly conceived). The term False Positive in relation to spam filtering means a message that was tagged as spam (with Declude this usually results from failure of multiple tests), but is in reality a legitimate email that needs to be delivered. Understandably there is some grey area in that, due to varying definitions on what email should be considered spam. For this reason, many admins' weighting systems vary on some of the details of implementation, due mostly to their user community, individual policies, and attempts to filter as much as possible without adversely affecting their community. I believe the point Scott was making was that the HELOBOGUS should not have much weight if you are seeing such a high percentage of emails (37%) that fail this particular test but are not spam. Darin. - Original Message - From: Dan Geiser [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:42 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System Hi, Scott, Thanks for the feedback. The more I thought about it after sending the e-mail a few minutes ago the more certain I was that my logic was not. in fact not even remotely close to being sound. It really has to be thought of as a factor of multiple tests and not just one, so I understand what you are saying. But I have to disagree with your terminology. I wasn't describing a false positive situation. I don't think the HELOBOGUS test by itself can have a false positive. A message either passes or fails the HELOBOGUS test. If a message fails the HELOBOGUS test, meaning the HELO is bogus by Scott's criterion, yet that message is not a spam message, i.e. it is a legit e-mail, it doesn't mean that the HELOBOGUS generated a false positive. The HELO either truly is BOGUS or NOT BOGUS. If HELOBOGUS misidentified a message as being BOGUS that was NOT BOGUS then, yes, I think that would be a false positive. But by it's nature one single test cannot create a false positive unless the program code for the test is written incorrectly. Just my thoughts. Dan - Original Message - From: Scott Fisher [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:20 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System If a test false positived 37% of the time, I certainly wouldn't be weighing it that high. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 12:57PM Hi, Markus, Thanks for responding. Well I went ahead and did it. I've rescaled everything to have 100 points be my HOLD weight. It was pretty easy because my previous HOLD weight was 5 so I just had to multiply everything by 20 to keep things relative. Now, that I have it there I would like to re-tune some of my weights. In your system, if you have a test like HELOBOGUS, for example, how do you decide what weight to give HELOBOGUS? I was thinking that if I had the correct statistics about which types of messages, spam or legit, were flagged by which tests it would be pretty straightforward. For example, if I knew that of 1000 messages that were flagged as failing the HELOBOGUS test and 37% of them were legit messages and 73% setting the weight of HELOBOGUS to 73 would be statistically sound. Is my thinking correct on that or am I way off base? Thanks, Dan Geiser [EMAIL PROTECTED] - Original Message - From: Markus Gufler To: [EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:25 PM Subject: RE: [Declude.JunkMail] Scaling Up The Declude Weighting System Dan, We've choosen to scale up the weighting system exactly for the two reasons you've mentioned below: -more granularity -absolute weight and percentage is the same Note that there are some good filter files maintained by other Declude users that are updated regulary and has the inside weights set up for a Hold-on-20 weighting system. Markus From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Geiser Sent: Tuesday, April 20, 2004 4:48 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] Scaling Up The Declude Weighting System Hello, All, Over the year or so that I've been reading the discussions on this list it seems I've read quite a bit about people scaling their weights up, i.e. instead of having a HOLD weight of 10, you might have a HOLD weight of 100 and then you adjust the corresponding test weights accordingly. Assuming that what I've read is correct, for those who uses this scaled up system... What sort of benefit is is that you feel that you receive from doing this? Does it allow a more granular tuning of your weighting system? Are there any other benefits I'm not thinking of? Does having a hold weight of 100, for example, help you think more clearly about each test being a percentage
Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
You guys are correct, I should have I shouldn't have said false positive with regards to the test. I just kept seeing the mostly good 37% of the mail 73% toward failing and false positives kept ringing in my head. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 02:10PM Dan, Individual tests do not false positive (unless they are poorly conceived). The term False Positive in relation to spam filtering means a message that was tagged as spam (with Declude this usually results from failure of multiple tests), but is in reality a legitimate email that needs to be delivered. Understandably there is some grey area in that, due to varying definitions on what email should be considered spam. For this reason, many admins' weighting systems vary on some of the details of implementation, due mostly to their user community, individual policies, and attempts to filter as much as possible without adversely affecting their community. I believe the point Scott was making was that the HELOBOGUS should not have much weight if you are seeing such a high percentage of emails (37%) that fail this particular test but are not spam. Darin. - Original Message - From: Dan Geiser [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:42 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System Hi, Scott, Thanks for the feedback. The more I thought about it after sending the e-mail a few minutes ago the more certain I was that my logic was not. in fact not even remotely close to being sound. It really has to be thought of as a factor of multiple tests and not just one, so I understand what you are saying. But I have to disagree with your terminology. I wasn't describing a false positive situation. I don't think the HELOBOGUS test by itself can have a false positive. A message either passes or fails the HELOBOGUS test. If a message fails the HELOBOGUS test, meaning the HELO is bogus by Scott's criterion, yet that message is not a spam message, i.e. it is a legit e-mail, it doesn't mean that the HELOBOGUS generated a false positive. The HELO either truly is BOGUS or NOT BOGUS. If HELOBOGUS misidentified a message as being BOGUS that was NOT BOGUS then, yes, I think that would be a false positive. But by it's nature one single test cannot create a false positive unless the program code for the test is written incorrectly. Just my thoughts. Dan - Original Message - From: Scott Fisher [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:20 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System If a test false positived 37% of the time, I certainly wouldn't be weighing it that high. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 12:57PM Hi, Markus, Thanks for responding. Well I went ahead and did it. I've rescaled everything to have 100 points be my HOLD weight. It was pretty easy because my previous HOLD weight was 5 so I just had to multiply everything by 20 to keep things relative. Now, that I have it there I would like to re-tune some of my weights. In your system, if you have a test like HELOBOGUS, for example, how do you decide what weight to give HELOBOGUS? I was thinking that if I had the correct statistics about which types of messages, spam or legit, were flagged by which tests it would be pretty straightforward. For example, if I knew that of 1000 messages that were flagged as failing the HELOBOGUS test and 37% of them were legit messages and 73% setting the weight of HELOBOGUS to 73 would be statistically sound. Is my thinking correct on that or am I way off base? Thanks, Dan Geiser [EMAIL PROTECTED] - Original Message - From: Markus Gufler To: [EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:25 PM Subject: RE: [Declude.JunkMail] Scaling Up The Declude Weighting System Dan, We've choosen to scale up the weighting system exactly for the two reasons you've mentioned below: -more granularity -absolute weight and percentage is the same Note that there are some good filter files maintained by other Declude users that are updated regulary and has the inside weights set up for a Hold-on-20 weighting system. Markus From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Geiser Sent: Tuesday, April 20, 2004 4:48 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] Scaling Up The Declude Weighting System Hello, All, Over the year or so that I've been reading the discussions on this list it seems I've read quite a bit about people scaling their weights up, i.e. instead of having a HOLD weight of 10, you might have a HOLD weight of 100 and then you adjust the corresponding test weights accordingly. Assuming that what I've read is correct, for those who uses this scaled up system
Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
I call them false positives, big whoop. I think people know what you mean :) Whatever you do though, don't mention women and spam in the same sentence!!! Matt Scott Fisher wrote: You guys are correct, I should have I shouldn't have said false positive with regards to the test. I just kept seeing the mostly good 37% of the mail 73% toward failing and "false positives" kept ringing in my head. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 02:10PM Dan, Individual tests do not "false positive" (unless they are poorly conceived). The term "False Positive" in relation to spam filtering means a message that was tagged as spam (with Declude this usually results from failure of multiple tests), but is in reality a legitimate email that needs to be delivered. Understandably there is some grey area in that, due to varying definitions on what email should be considered spam. For this reason, many admins' weighting systems vary on some of the details of implementation, due mostly to their user community, individual policies, and attempts to filter as much as possible without adversely affecting their community. I believe the point Scott was making was that the HELOBOGUS should not have much weight if you are seeing such a high percentage of emails (37%) that fail this particular test but are not spam. Darin. - Original Message - From: "Dan Geiser" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:42 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System Hi, Scott, Thanks for the feedback. The more I thought about it after sending the e-mail a few minutes ago the more certain I was that my logic was not. in fact not even remotely close to being sound. It really has to be thought of as a factor of multiple tests and not just one, so I understand what you are saying. But I have to disagree with your terminology. I wasn't describing a false positive situation. I don't think the HELOBOGUS test by itself can have a false positive. A message either passes or fails the HELOBOGUS test. If a message fails the HELOBOGUS test, meaning the HELO is bogus by Scott's criterion, yet that message is not a spam message, i.e. it is a legit e-mail, it doesn't mean that the HELOBOGUS generated a false positive. The HELO either truly is BOGUS or NOT BOGUS. If HELOBOGUS misidentified a message as being BOGUS that was NOT BOGUS then, yes, I think that would be a false positive. But by it's nature one single test cannot create a false positive unless the program code for the test is written incorrectly. Just my thoughts. Dan - Original Message - From: "Scott Fisher" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:20 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System If a test false positived 37% of the time, I certainly wouldn't be weighing it that high. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 12:57PM Hi, Markus, Thanks for responding. Well I went ahead and did it. I've rescaled everything to have 100 points be my HOLD weight. It was pretty easy because my previous HOLD weight was 5 so I just had to multiply everything by 20 to keep things relative. Now, that I have it there I would like to re-tune some of my weights. In your system, if you have a test like HELOBOGUS, for example, how do you decide what weight to give HELOBOGUS? I was thinking that if I had the correct statistics about which types of messages, spam or legit, were flagged by which tests it would be pretty straightforward. For example, if I knew that of 1000 messages that were flagged as failing the HELOBOGUS test and 37% of them were legit messages and 73% setting the weight of HELOBOGUS to 73 would be statistically sound. Is my thinking correct on that or am I way off base? Thanks, Dan Geiser [EMAIL PROTECTED] - Original Message - From: Markus Gufler To: [EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:25 PM Subject: RE: [Declude.JunkMail] Scaling Up The Declude Weighting System Dan, We've choosen to scale up the weighting system exactly for the two reasons you've mentioned below: -more granularity -absolute weight and percentage is the same Note that there are some good filter files maintained by other Declude users that are updated regulary and has the "inside" weights set up for a Hold-on-20 weighting system. Markus From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Dan Geiser Sent: Tuesday, April 20, 2004 4:48 PM To: [EMAIL PROTECTED] Subject: [Declude.JunkMail] Scaling Up The Declude Weighting System Hello, All,
Re: [Declude.JunkMail] Scaling Up The Declude Weighting System
Guess we can't sing Monty Python songs then, can we? Darin. - Original Message - From: Matt To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 3:58 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System I call them false positives, big whoop. I think people know what you mean :)Whatever you do though, don't mention women and spam in the same sentence!!!MattScott Fisher wrote: You guys are correct, I should have I shouldn't have said false positive with regards to the test. I just kept seeing the mostly good 37% of the mail 73% toward failing and "false positives" kept ringing in my head. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 02:10PM Dan, Individual tests do not "false positive" (unless they are poorly conceived). The term "False Positive" in relation to spam filtering means a message that was tagged as spam (with Declude this usually results from failure of multiple tests), but is in reality a legitimate email that needs to be delivered. Understandably there is some grey area in that, due to varying definitions on what email should be considered spam. For this reason, many admins' weighting systems vary on some of the details of implementation, due mostly to their user community, individual policies, and attempts to filter as much as possible without adversely affecting their community. I believe the point Scott was making was that the HELOBOGUS should not have much weight if you are seeing such a high percentage of emails (37%) that fail this particular test but are not spam. Darin. - Original Message - From: "Dan Geiser" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:42 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System Hi, Scott, Thanks for the feedback. The more I thought about it after sending the e-mail a few minutes ago the more certain I was that my logic was not. in fact not even remotely close to being sound. It really has to be thought of as a factor of multiple tests and not just one, so I understand what you are saying. But I have to disagree with your terminology. I wasn't describing a false positive situation. I don't think the HELOBOGUS test by itself can have a false positive. A message either passes or fails the HELOBOGUS test. If a message fails the HELOBOGUS test, meaning the HELO is bogus by Scott's criterion, yet that message is not a spam message, i.e. it is a legit e-mail, it doesn't mean that the HELOBOGUS generated a false positive. The HELO either truly is BOGUS or NOT BOGUS. If HELOBOGUS misidentified a message as being BOGUS that was NOT BOGUS then, yes, I think that would be a false positive. But by it's nature one single test cannot create a false positive unless the program code for the test is written incorrectly. Just my thoughts. Dan - Original Message - From: "Scott Fisher" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 2:20 PM Subject: Re: [Declude.JunkMail] Scaling Up The Declude Weighting System If a test false positived 37% of the time, I certainly wouldn't be weighing it that high. Scott Fisher Director of IT Farm Progress Companies [EMAIL PROTECTED] 04/22/04 12:57PM Hi, Markus, Thanks for responding. Well I went ahead and did it. I've rescaled everything to have 100 points be my HOLD weight. It was pretty easy because my previous HOLD weight was 5 so I just had to multiply everything by 20 to keep things relative. Now, that I have it there I would like to re-tune some of my weights. In your system, if you have a test like HELOBOGUS, for example, how do you decide what weight to give HELOBOGUS? I was thinking that if I had the correct statistics about which types of messages, spam or legit, were flagged by which tests it would be pretty straightforward. For example, if I knew that of 1000 messages that were flagged as failing the HELOBOGUS test and 37% of them were legit messages and 73% setting the weight of HELOBOGUS to 73 would be statistically sound. Is my thinking correct on that or am I way off base? Thanks, Dan Geiser [EMAIL PROTECTED] - Original Message - From: Markus Gufler To: [EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:25 PM Subject: RE: [Declude.JunkMail] Scaling Up The Declude Weighting System Dan, We've choosen to scale up the weighting system exactly for the two reasons you've mentioned below: -more granularity -absolute weight and percentage is the same Note that there are some good filter files maintained by other Declude users that are updated regulary and has the "inside" weights set up for a Hold-on-20 weighting system. Markus From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Be
RE: [Declude.JunkMail] Scaling Up The Declude Weighting System
I think it's not possible to calculate the weight of an individual test strictly from his catch/failure rate. On http://www.zcom.it/spamtest/you can see what we generate from our daily logfiles. In my opinion it's not enough to count wrong or right results. Theoretically there are 5 possible results for every individual test correct result for a spam messageFor example SPAMCOP has a positive result for a spam message wrong result for a spam messageFor example NOLEGITCONTENT has a positive result (and so will substract points) for a spam message correct result for a legit messageFor example AUTOWHITE has a positive result (and so will substract points) for a legit message wrong result for a legit messageFor example REVDNS has a positive result for a legit message no resultFor example no line ina FILTER file matches with something in the legit or spam message Practically most spam tests has only 3 possible results because they are counting "only" or as positive or as negative test. For example SPAMCOP can't fail on a spam message because his result is a "positive weight" or "no weight" (unless you decide to assign a negative weight if spamcop hasn't a positive result = not considered) Another test like NOLEGITCONTENT will only substract points or if NO-LEGIT-CONTENT was found return zero as result. Some tests like SPAMCHK can have a positive/negative weight or zero as result and so he can have all 5 results mentioned above. On the report (link above) you can see this 5 possible results both in absolute numbers or as relative values in the diagramm: dark green dark red light green light red grey The more green you can see, the bether a test is. The red bars indicate that this test has counted in the opposite direction as the final weight. (You can move the mouse pointer above the bar to show the percentage.) If a certain test has no false positives over several days, weeks or months you can increase his weight near to your hold weight or also above. But this tests are very rare. Good tests has a good detection rate, and very few false positves. for example SPAMCOP. My scripts, applications and the database for all this research is a work in progress and I have a lot of ideas to implement. For example I've added a report to view mail-from, -to and subject for every message where a certain test has had the wrong result. So I can see if this test if failing has some effect or can be ignored. The report above shows the result for one business day. But I can also create average values for several days or weeks. The next thing I plan is to create a diagram containing the daily results for one single test. So I can see if the quality of this test changes over time (goes up, down, ...) and so the weight should be adapted. Unfortunately I can't code this into a redistribuable application. My VBscripts are not very fast (would be much faster without error checking for corrupt logfile lines) and parsing trough 10 MB logfiles, analizing the individual results, saving them into a database (MS-SQL Server) and creating all necessary conjuntions takes several minutes with high CPU usage. I'm sure a good programmer and compiler can code this in a small and fast application. But at the moment I see this as a research what's worth analizing and searching for. Finaly some comments to previous posts: 37% as way too much. Even if the resting 63% (not 73% Scott :-) are correct results. Remove this test! Some "old" test like REVDNS or HELOBOGUS seem sto have an unexpected high rate of wrong results. I've decreased their weight since I've discovered this. regaring the terminology of false positives: I agree with Dan, that a single test can't create a false positive (unless his own weight is superior then the HOLD weight) So a test failing in his result should be interpreted as "wrong result". The"False positive"is a legit message in your spamfolder. The "False negative" is a spam message in your mailbox. Hope my "english" is not too terrible ;-) Markus
RE: [Declude.JunkMail] Scaling Up The Declude Weighting System
Title: Message This is the weighting that I use: Hold Weight = 10 Delete Weight = 20 9: SNIFFER2 8: BADHEADERS 7: BLITZEDALL SBL SPAMCOP COMMENTS 6: SPAM-DOMAINS AHBL DSBL 5: ORDB SORBS-HTTP SORBS-SOCKS SORBS-MISC SORBS-SMTP SORBS-SPAM SORBS-WEB SORBS-ZOMBIE SORBS-DUHL 4: MAILFROM CBL BASE64 REVDNS ROUTING SPFFAIL 3: DSN HOUR (12AM - 6AM) SPAMHEADERS 2: NOABUSE NOPOSTMASTER -5 BONDEDSENDER SPFPASS For filters, I normally will use a 9 unless it is a new one that I am testing. I end up with a hold percentage of about 93% and a delete of about 89%. -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan GeiserSent: Tuesday, April 20, 2004 9:48 AMTo: [EMAIL PROTECTED]Subject: [Declude.JunkMail] Scaling Up The Declude Weighting System Hello, All, Over the year or so that I've been reading the discussions on this list it seems I've read quite a bit about people scaling their weights up, i.e. instead of having a HOLD weight of 10, you might have a HOLD weight of 100 and then you adjust the corresponding test weights accordingly. Assuming that what I've read is correct, for those who uses this scaled up system... What sort of benefit is is that you feel that you receive from doing this? Does it allow a more granular tuning of your weighting system? Are there any other benefits I'm not thinking of? Does having a hold weight of 100, for example, help you think more clearly about each test being a percentage of the overall HOLD weight? I'm doing a major overhaul of Declude JunkMail configuration and I figured if a scaled up weight system is the best way to do things then I might want to implement that now. Thanks In Advance For Your Comments! Dan Geiser [EMAIL PROTECTED]
