I'm always correcting HMM/Bayes by reviewing the block report on a daily
basis and reporting.  It's a horrible task that I dread, but it is worth
it.  I have the score set to 50, and 50 as the threshold for rejection.
We've tried lower, but too much spam is only tagged *solely* due to a HMM
hit and slips through.

I never changed baysConf from the original 0.001 with baysprobablity to
0.6, but I do have a 1.000 corpus norm.   It's certainly a mature
installation (15+ years).  The issue I'm having is legit messages, say a
message advertising handbags from a reputable seller that one of our staff
buys from (over their lunch hour of course!!).  Then there's a ton of spam
messages also advertising (fake) handbags and they're often using the SAME
domain and sometimes identical from address as the legit mails.  HMM/Bayes
is rightfully biased against all handbag email.  You're saying that
increasing baysConf will help the legit ones get through but still will
block the spam ones (with almost identical content)?   I've read through
the gui for baysConf, but the problem is that which I understand HMM and
Bayes from a concept standpoint, the calculations aren't something I
understand, so I don't dare change the 0.0001 threshold without real
guidance from you.  I understand 1 is the max, but don't know if increases
linearly or exponentially change levels, don't know if a change from 0.001
to 0.005 would be significant or make sense, etc.

My hopes with the dkim np was to let hmm spammy mail through if it's a dkim
match.  Ignore all other results, if the DKIM is good, just let it through
was my thinking.  I know handbag seller X sends ad mail that's DKIM signed,
but I don't know when IP they'll come from (or the IP is mailing service
that I don't want to blanket allow).  This has been quite successful with a
whole lot of mail.  I've become spoiled, now I want it to work for all mail
when there's a DKIM match to the NP list.

You raise another a good point about the 2 kind of DKIM checks, thanks for
the reminder.  Does it make any sense to *always* have ASSP do the second
one and if it validates and matches dkimNPaddress or dkimWLadderss, process
solely based on that match?  For example, HMM might hit before the full
body validation of DKIM, but so what?  If we put the address on the
DKIMNPAddress list, shouldn't it honor that regardless of anything else?
NO processing, as I'm interpreting it, should mean, well, NONE, so if other
hits have happened, they're ignored because we said don't process.

I will temporarily change the logging level for a bit and see if I can
figure out why dkim isn't being done for these messages, but I'm guessing
that it's by design.

As always, thanks
Ken


On Mon, Apr 16, 2018 at 2:35 AM, Thomas Eckardt <thomas.ecka...@thockar.com>
wrote:

> >I'm still seeing scenarios where analyze shows a DKIM NP match, but the
> message is still going to spam based on score from HMM.
>
> The best solution is to correct the corpus, to get better HMM results. You
> may also decrease the scoring points for HMM and/or Bayesian. If the corpus
> is corrected and the corpusnorm is ~ 1.0 , 'baysConf' will increase
> detection correctness.
>
>
> If assp receives a mail it acts as a statemachine. If and how a check is
> done, depends on the previouse reached states.
>
> Using the analyzer, assp acts procedural. Every check is done without any
> state dependency. This is done, to be able to show every feature match.
> The analyzer uses the current configuration, hashes, lists and databases.
> So it may be normal to get different results compared to the real mail
> processing loggings, if a mail is analyzed.
>
> >DKIM NP match:
>
> The analyzer checks DKIM without any dependency and shows all results.
>
> But, if a mail is received, the DKIM check depends on several previouse
> states
> DKIM NP is a resulting state of the DKIM check. So - if any of the
> previouse (DKIM depdency) states prevents the DKIM check, there will be no 
> DKIM
> (DKIM NP) result. The mail will be processed the same way, as it was not
> DKIM signed.
> Every state, that depends on DKIM NP will not be reached.
>
> You should also remember, that assp use two DKIM checks. The full DKIM
> check, which requires the full mail to be received - the results of this
> check affects only the Plugin Level 2 (full mail) checks.
> And the DKIM-Pre-Check - which is done after the MIME header is received
> and if 'DKIMCacheInterval' is enabled. The results (states) of this check
> affects most of the header checks and all body and full mail checks.
>
> If you expect but miss a match for 'DKIMWLAddresses' or 'DKIMNPAddresses'
> , you may increase the logging level (ValidateSenderLog, SessionLog,
> ipmatchLogging, slmatchLogging).
>
> Thomas
>
>
>
>
> Von:        "K Post" <nntp.p...@gmail.com>
> An:        "ASSP development mailing list" <assp-test@lists.sourceforge.
> net>
> Datum:        15.04.2018 23:21
> Betreff:        Re: [Assp-test] Analyze shows DKIMNPAddress match as
> expected, but some messages still processed as spam?
> ------------------------------
>
>
>
> I'm still seeing scenarios where analyze shows a DKIM NP match, but the
> message is still going to spam based on score from HMM.
>
> On Mon, Apr 9, 2018 at 12:19 PM, K Post <*nntp.p...@gmail.com*
> <nntp.p...@gmail.com>> wrote:
> cheers.
>
> On Mon, Apr 9, 2018 at 3:36 AM, Thomas Eckardt <
> *thomas.ecka...@thockar.com* <thomas.ecka...@thockar.com>> wrote:
> If assp has modified the original mail header (changed foreign X-ASSP- or
> removed cc,bcc or ...) the DKIM check is skipped and also DKIMNPAddress and
> DKIMWLAddress.
>
> The next version will try to check, if removed or changed headers are
> protected by a DKIM signature and do the check, if this is not the case.
>
> Thomas
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! *http://sdm.link/slashdot*
> <http://sdm.link/slashdot>
> _______________________________________________
> Assp-test mailing list
> *Assp-test@lists.sourceforge.net* <Assp-test@lists.sourceforge.net>
> *https://lists.sourceforge.net/lists/listinfo/assp-test*
> <https://lists.sourceforge.net/lists/listinfo/assp-test>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Assp-test mailing list
> Assp-test@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test

Reply via email to