Re: [Declude.JunkMail] Bayesian filtering

2002-09-23 Thread R. Scott Perry


Has anyone seen anything about this .. Any good or bad ? .. Bayesian
filtering .. Check this link http://www.paulgraham.com/filters.html

Scott are you thinking along any of these line for the future of
Declude?

Here's part of an E-mail that I posted previously:

---

The concept is flawed, but sounds like it could work well if implemented 
properly.  It's essentially the same thing that the heuristics test (which 
never made it out of beta) does, except that the heuristics test only looks 
at about 100 different aspects of the E-mail (not necessarily individual 
words).

The flaw is that it is based on the Bayes Rule, but doesn't meet its 
conditions (the individual probabilities used must not be related to each 
other, but they are in this situation).  For example, if you have 10 spams 
that each have the term This is a spam in them, and it's determined that 
each word has a 99% chance of indicating spam (IE if the word This 
appears in an E-mail, there's a 99% chance it is spam), the Bayes Rule 
would say that an E-mail with This is a spam would have about a 99.9% 
chance of being spam (when in reality, I believe it would only have a 99% 
chance of being spam).  This probability stuff gets pretty complex unless 
you've taken a probability and statistics course recently (and did well in it).

Another problem is that you need a good database of legitimate E-mail to 
work with, which is harder to obtain than a good database of spam (which 
isn't always easy to get).  If you just use E-mail to a specific person, it 
will skew the results.

We are doing some testing here to see if the concept could work well (as it 
seems to for Paul Graham).
-Scott

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.



[Declude.JunkMail] Bayesian filtering

2002-09-22 Thread Brian R. Watters

Hello all,

Has anyone seen anything about this .. Any good or bad ? .. Bayesian
filtering .. Check this link http://www.paulgraham.com/filters.html

Scott are you thinking along any of these line for the future of
Declude?

Brian



---
[Scanned for viruses with safE-Mail by American Broadband Services]

[To scan your Companies email contact ABS [EMAIL PROTECTED]]
---

---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]

---
This E-mail came from the Declude.JunkMail mailing list.  To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type unsubscribe Declude.JunkMail.  The archives can be found
at http://www.mail-archive.com.