Collin,
back to start - I think this was the short mail with the UTF-8 BOM and the
single link.
put the following in bombDataRe (in a single line)
^\s*[\S\x80-\xFF]{0,3}\s*(?:<\/?(?:html\s*|head\s*|meta[^>]*)>)+\s*\<\s*body\s*>\s*(?:ht|f)tps?:\/\/[\w\.\/\-\?\&\=]+<\s*\/\s*body[^>]*>\s*<\s*\/html\s*>[
\t\f]*\s*.{0,10}$
switch off 'DoTransliterate' otherwise the regex will not match
Thomas
Von: "Colin Waring" <co...@lanternhosting.co.uk>
An: "'ASSP development mailing list'"
<assp-test@lists.sourceforge.net>,
Datum: 31.01.2014 11:03
Betreff: Re: [Assp-test] Bayes mistake
Hi Thomas,
Turns out another one got through the spam filtering yesterday evening.
Again same message content.
We have it the way it is because HMM misses smaller messages, we can't put
either one to a higher weight otherwise we end up with more false
positives.
I'd love to turn off Bayes and just use HMM but it isn't worth it for the
complaints on the short messages spam that gets through.
All the best,
Colin Waring.
-----Original Message-----
From: Thomas Eckardt [mailto:thomas.ecka...@thockar.com]
Sent: 31 January 2014 07:04
To: ASSP development mailing list
Subject: Re: [Assp-test] Bayes mistake
Two reasons:
>I hadn't reported the previous one as a false negative yet.
1) Another one has reported the same or similar mail. ASSP V2 recalculates
the Bayes and HMM database on the fly if a mail is reported
2) A rebuild was done.
>Is there any way to figure out why Bayes made a boob on the first one?
No - all checks are done on the current DB's - no chance to go back in the
past. But I think, after eliminating pairs of very low (ham) and very high
(spam) values, there was at least one very low value left.
If you use both HMM and Bayes - set the scoring so, that your trust on HMM
is higher. Bayes is fine but less exact - for this reason HMM was
implemented.
Thomas
Von: "Colin Waring" <co...@lanternhosting.co.uk>
An: "'ASSP development mailing list'"
<assp-test@lists.sourceforge.net>,
Datum: 30.01.2014 21:15
Betreff: [Assp-test] Bayes mistake
Hi there,
I'm wondering what's the best way to troubleshoot a Bayes mistake. We get
tonnes of fake bank security alert emails and nearly all of them got
blocked.
Imagine my surprise to see one in my own inbox this morning from
barcl...@email.barclays.co.uk <mailto:barcl...@email.barclays.co.uk>
So I checked the logs. What I found was more surprising. The exact same
message with the exact same content (I compared the .eml files and only
the
headers were different) hit my server later on and was blocked by Bayes. I
hadn't reported the previous one as a false negative yet.
Is there any way to figure out why Bayes made a boob on the first one?
Cheers,
Colin.
2014-01-30 09:41:52 m1-74904-00342 [Worker_4] [TLS-in] [TLS-out]
212.227.137.50 <barcl...@email.barclays.co.uk> to: m...@mydomain.tld HMM
Check
[scoring] - Prob: 1.00000 => spam
2014-01-30 09:41:52 m1-74904-00342 [Worker_4] [TLS-in] [TLS-out]
212.227.137.50 <barcl...@email.barclays.co.uk> to: m...@mydomain.tld
Message-Score: added 20 for HMM Probability: 1.0000, total score for this
message is now 35
2014-01-30 09:41:53 m1-74904-00342 [Worker_4] [TLS-in] [TLS-out]
212.227.137.50 <barcl...@email.barclays.co.uk> to: m...@mydomain.tld
Bayesian
Check [scoring] - Prob: 0.10750 => ham
2014-01-30 12:40:56 m1-85654-02281 [Worker_7] [TLS-out] 85.94.77.22
<barcl...@email.barclays.co.uk> to: m...@mydomain.tld HMM Check [scoring] -
Prob: 1.00000 => spam
2014-01-30 12:40:56 m1-85654-02281 [Worker_7] [TLS-out] 85.94.77.22
<barcl...@email.barclays.co.uk> to: m...@mydomain.tld Message-Score: added
20
for HMM Probability: 1.0000, total score for this message is now 40
2014-01-30 12:40:56 m1-85654-02281 [Worker_7] [TLS-out] 85.94.77.22
<barcl...@email.barclays.co.uk> to: m...@mydomain.tld Bayesian Check
[scoring]
- Prob: 0.99597 => spam
2014-01-30 12:40:56 m1-85654-02281 [Worker_7] [TLS-out] 85.94.77.22
<barcl...@email.barclays.co.uk> to: m...@mydomain.tld Message-Score: added
30
for Bayesian Probability: 0.99597, total score for this message is now 70
----------------------------------------------------------------------------
--
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally
privileged and protected in law and are intended solely for the use of the
individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no
known virus in this email!
*******************************************************
------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally
privileged and protected in law and are intended solely for the use of the
individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no
known virus in this email!
*******************************************************
------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
