Your explanations are invaluable. I look forward to reviewing the next release for the improvements and can't wait to see how our corpus improves now that HMM is being used.
Thank you again. I could go on discussing forever - thanks for stopping me. On Mon, May 4, 2015 at 1:39 PM, Thomas Eckardt <thomas.ecka...@thockar.com> wrote: > >Why wouldn't we > >want the score be reduced for low confidence? > > because the PenaltyBox is working this way - every feature sets, counts, > calculates or weights results to resulting scores > > Calculating/weighting scores base on the Bayes/HMM results is still > implemented. The next build will use a better code for this. > > .... > The Bayesian check will only run, if the spam/ham probability of the HMM > check is in a given value range or the HMM check has given too few results > or > the confidence ( baysConf ) of the detection is too low. > .... > > > How ever you need to configure 'baysConf'! assp will get a probability in > every case, but this says nothing about the confidence. > > Here are two extreme examples: > > you can get a probability of 0.650 with a confidence of 0.5 - which is > 100% spam - no doubt > and > you can get a probability of 0.999 with a confidence of 0.00001 - which is > a very wild guess (50.00001%(spam) to 49.99999%(ham)) > > 'baysConfidenceHalfScore' will reduce the score to at least the half for > both checks - so the result will be max. the full. > > >is correct math, but where do the 12 and 3 numbers come from? > > I don't want to discuss the Bayesian/HMM math. These are simple numbers, > everyone will understand them - less results -> less confidence. > ASSP would not need these numbers, if everyone has a correct corpus > (corpusnorm). These numbers are something like milestones, if they are > reached, the code loses trust to the math of the confidence and switches > the general behavior (block -> score -> monitor). > > I stop this discussion here. Both features are working nearly perfect for > years now - in my case with less than 0.5% correction rate for each and > less than 0.25% together, if the corpus is correct. > > Thomas > > > > > > Von: K Post <nntp.p...@gmail.com> > An: ASSP development mailing list <assp-test@lists.sourceforge.net> > Datum: 04.05.2015 18:28 > Betreff: Re: [Assp-test] HMM and Bayesian Running > > > > This is now starting to make more sense to me. Thank you for the > explanations. > > 6 = 60 / 12 +1 > 21 = 60 / 3 +1 > is correct math, but where do the 12 and 3 numbers come from? > > My BayConf is 0, and intentionally so because my norm is 1.0000 or very > close to that. That's correct logic right? > And yes, bayesProbability is 0.60 > > I guess my original question still stands: Why not only count EITHER hmm > or > bayes if confidence is low on both? And why would "the confidence is very > low - but it is ignored for the resulting score" be true. Why wouldn't we > want the score be reduced for low confidence? > > Under BayesAfterHMM is says: > The Bayesian check will only run, if the spam/ham probability of the HMM > check is in a given value range or the HMM check has given too few > results. > > Is there an equivalent, easy to implement, and *sensible* solution that is > something like: > HMMScoreDiscardIfOutsideOfRange: > The HMM score will only be counted if the HMM value range is outside of > this range. Setting this range to something like 0.35-0.65, when paired > with BayesAfterHMM (which will allow Bayesian to run after an HMM check) > stops the HMM score from being added to the bayesian score which would > result in an unreasonably high score. > > I'm kind of just thinking out loud here. Please tell me if what I'm > suggesting doesn't make sense or if I need to be more clear. > > On Mon, May 4, 2015 at 3:50 AM, Thomas Eckardt > <thomas.ecka...@thockar.com> > wrote: > > > >In your example, how is the 6 or 21 number calculated based on a > > >maxBayesValues of 60? > > > > 6 = 60 / 12 +1 > > 21 = 60 / 3 +1 > > > > >I've got BayesAfterHMM set to 0.3-0.7 > > > > I assume 'baysProbability' is set to 0.6 - and 'baysConf' is set to zero > - > > right ? > > > > If assp got at least 6 but less than 21 results and the resulting HMM > > probability is for example 0.65 : > > > > - DoHMM is set to score for this mail > > - Bayes will run, because 0.65 is less than 0.7 and more than 0.3 > > - the confidence is very low - but it is ignored for the resulting score > > - without 'baysConf', 0.65 is 100% SPAM because 'baysProbability' is > set > > to 0.6 > > > > The mail gets the full HMM score + the full Bayesian score. > > > > > > Thomas > > > > > > > > > > Von: K Post <nntp.p...@gmail.com> > > An: ASSP development mailing list <assp-test@lists.sourceforge.net> > > Datum: 03.05.2015 22:15 > > Betreff: Re: [Assp-test] HMM and Bayesian Running > > > > > > > > In your example, how is the 6 or 21 number calculated based on a > > maxBayesValues of 60? > > > > And more importantly, with my example, where there was 49 scored because > > of > > HMM, and then another 49 scored for bayesian, also which wasn't > > particularly certain. Net result: a really high score - not really > > accurate. Is there some why to tell ASSP that if HMM and Bayes both > > assign > > scores, and HMM is based on a really low number of results, just ignore > > the > > HMM result? I feel like that would be more accurate. > > > > On Sun, May 3, 2015 at 2:55 AM, Thomas Eckardt > > <thomas.ecka...@thockar.com> > > wrote: > > > > > It works this way. > > > > > > For example, if 'maxBayesValues' is set to 60 > > > > > > - DoHMM is set to monitor if less than 6 results are found -> Bayes > will > > > run > > > - DoHMM is set to score, if DoHMM is configured to block and less than > > 21 > > > results are found - Bayes will run on 'BayesAfterHMM' > > > > > > Thomas > > > > > > > > > > > > > > > > > > > > > > > > > > > Von: K Post <nntp.p...@gmail.com> > > > An: ASSP development mailing list > <assp-test@lists.sourceforge.net> > > > Datum: 02.05.2015 16:34 > > > Betreff: [Assp-test] HMM and Bayesian Running > > > > > > > > > > > > I've got BayesAfterHMM set to 0.3-0.7 > > > > > > Sometimes, I see bayesian run even though HMM scores a probability of > > > 1.00000. Is that right? Could this be happening because HMM doesn't > > have > > > enough hits? If that's the case can we tell ASSP to ignore HMM it's > > > unsure > > > and use Bayesian INSTEAD (only) in these cases? > > > > > > Sorry for the HMM questions, but the concept's new to me. > > > > > > > > > Clearly my databases aren't quite perfect, but I was hoping that > > > senderbase > > > would bring the HMM score down enough to pass it. > > > > > > However, with Bayesian running too, we exceed the max and it's > blocked. > > > > > > X-ASSP-Message-Score: -35 (White Organization/Domain > > > 'RESPONSYS') > > > X-ASSP-IP-Score: -34 (White Organization/Domain 'RESPONSYS') > > > X-ASSP-Detected-URI: rsys2.com(1), e.hotwire.com(1), > > > hotwire.com(7) > > > X-ASSP-Message-Score: 49 (HMM Probability: 1.0000) > > > X-ASSP-IP-Score: 49 (HMM Probability: 1.0000) > > > X-ASSP-Message-Score: 49 (Bayesian Probability: 1.00000) > > > X-ASSP-IP-Score: 49 (Bayesian Probability: 1.00000) > > > X-ASSP-Spam-Prob: 1.00000 > > > X-ASSP-HMM-Spam-Prob: 1.00000 > > > X-ASSP-Tag: MessageLimit > > > X-ASSP-Spam-Reason: MessageScore 63, limit 50 > > > X-ASSP-Message-Totalscore: 63 > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > One dashboard for servers and applications across > Physical-Virtual-Cloud > > > Widest out-of-the-box monitoring support with 50+ applications > > > Performance metrics, stats and reports that give you Actionable > Insights > > > Deep dive visibility with transaction tracing using APM Insight. > > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > > _______________________________________________ > > > Assp-test mailing list > > > Assp-test@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > > > > > > > > > > > > DISCLAIMER: > > > ******************************************************* > > > This email and any files transmitted with it may be confidential, > > legally > > > privileged and protected in law and are intended solely for the use of > > the > > > > > > individual to whom it is addressed. > > > This email was multiple times scanned for viruses. There should be no > > > known virus in this email! > > > ******************************************************* > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > One dashboard for servers and applications across > Physical-Virtual-Cloud > > > Widest out-of-the-box monitoring support with 50+ applications > > > Performance metrics, stats and reports that give you Actionable > Insights > > > Deep dive visibility with transaction tracing using APM Insight. > > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > > _______________________________________________ > > > Assp-test mailing list > > > Assp-test@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > ------------------------------------------------------------------------------ > > One dashboard for servers and applications across Physical-Virtual-Cloud > > Widest out-of-the-box monitoring support with 50+ applications > > Performance metrics, stats and reports that give you Actionable Insights > > Deep dive visibility with transaction tracing using APM Insight. > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > _______________________________________________ > > Assp-test mailing list > > Assp-test@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > > > > > > > > DISCLAIMER: > > ******************************************************* > > This email and any files transmitted with it may be confidential, > legally > > privileged and protected in law and are intended solely for the use of > the > > > > individual to whom it is addressed. > > This email was multiple times scanned for viruses. There should be no > > known virus in this email! > > ******************************************************* > > > > > > > > ------------------------------------------------------------------------------ > > One dashboard for servers and applications across Physical-Virtual-Cloud > > Widest out-of-the-box monitoring support with 50+ applications > > Performance metrics, stats and reports that give you Actionable Insights > > Deep dive visibility with transaction tracing using APM Insight. > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > _______________________________________________ > > Assp-test mailing list > > Assp-test@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > ------------------------------------------------------------------------------ > One dashboard for servers and applications across Physical-Virtual-Cloud > Widest out-of-the-box monitoring support with 50+ applications > Performance metrics, stats and reports that give you Actionable Insights > Deep dive visibility with transaction tracing using APM Insight. > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > ------------------------------------------------------------------------------ > One dashboard for servers and applications across Physical-Virtual-Cloud > Widest out-of-the-box monitoring support with 50+ applications > Performance metrics, stats and reports that give you Actionable Insights > Deep dive visibility with transaction tracing using APM Insight. > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > ------------------------------------------------------------------------------ One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
