>Jan-28-16 09:28:22 Spam Weight : 3,264,527
>Jan-28-16 09:28:22 Not-Spam Weight: 3,265,258
>Jan-28-16 09:28:22 Corpus norm: 0.9998 - (very good - balanced)
>Jan-28-16 09:28:22 Corpus confidence: 0.06250000
Corpus confidence: 0.06250000 - this value is impossible (expected is
1.000) if -> Corpus norm: 0.9998 - (very good - balanced).
I think your Berkeley-DB ENV or DB is damaged for some or all BDB files -
but at least for HMMdb.
- shutdown assp
- remove all files (__*.* , *.bdb) from assp/tmpDB/HMMdb
- do the same for spamdb
- remove assp/hmmdb.bdb and assp/spamdb.bdb
- start assp
- import any avalable backup for both DB's - or run a rebuildspamdb
- restart assp to force a recalculation of the used BDB cache
If you use any *nix - KEEP in MIND! Your init.d script for assp (stop
case) has to wait until assp has been finished - otherwise your BerkeleyDB
files WILL BE DESTROYED!
To be clear - I mean 'WILL BE DESTROYED' - not 'may be' or 'possibly' Some
damaging of BDB files is fixed by an assp internal BDB repair mechanism -
some , not all!
Thomas
Von: Dossy Shiobara <do...@panoptic.com>
An: assp-user@lists.sourceforge.net
Datum: 28.01.2016 17:17
Betreff: [Assp-user] HMM-Check has given less than 6 results -
using monitoring mode only
Recently, it seems my HMM and Bayes checks are no longer working? In
mail log, I see:
"HMM-Check has given less than 6 results - using monitoring mode only"
I'll include my latest rebuildrun.txt, which looks like it ran
successfully.
Why is this happening? I'm running ASSP 2.4.7(16004). Also, it seems
like if I get this error, it doesn't even perform Bayesian scoring --
basically, spam that was previously being blocked is now being let
through...
---rebuildrun.txt---
Jan-28-16 09:05:00 RebuildSpamDB-thread rebuildspamdb-version 7.26
started in ASSP version 2.4.7(16004)
Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB for temporary hashes
Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB-ENV with 62.50 MByte
Jan-28-16 09:05:00 RebuildSpamDB will create a Hidden Markov Model
Jan-28-16 09:05:00 RebuildSpamDB will create unicode enabled databases
Jan-28-16 09:05:00 RebuildSpamDB will process all words as Sequence of
UAX #29 Grapheme Clusters
Jan-28-16 09:05:00 RebuildSpamDB will normalize unicode characters
Jan-28-16 09:05:00 RebuildSpamDB will use the ASSP_WordStem engine
Jan-28-16 09:05:00 ---ASSP Settings---
Jan-28-16 09:05:00 Do Not Collect Messages with RedListed address: Enabled
**Messages with RedListed addresses will be removed from the corpus!**
Jan-28-16 09:05:00 Do Not Collect RedRe Messages: Enabled
**Messages matching the RedRe will be removed from the corpus!**
Jan-28-16 09:05:00 Use Subject as Maillog Names: True
Jan-28-16 09:05:00 Maxbytes: 4,000
Jan-28-16 09:05:00 RebuildFileTimeLimit: 1 5
Jan-28-16 09:05:00 RebuildFileTimeLimit: files will be moved away from
the corpus if their processing takes longer than 5 second(s)
Jan-28-16 09:05:00 /data/assp/errors/spam
Jan-28-16 09:05:00 File Count: 11
Jan-28-16 09:05:00 Processing... errors/spam with 11 files
Jan-28-16 09:05:00 ignore and remove files older than Sep-11-88 10:05:00
in folder errors/spam
Jan-28-16 09:05:01 Imported Files for HeloBlackList: 10
Jan-28-16 09:05:01 Imported Files for Bayes/HMM: 10
Jan-28-16 09:05:01 Finished in 1 second(s)
Jan-28-16 09:05:01 /data/assp/errors/notspam
Jan-28-16 09:05:01 File Count: 1
Jan-28-16 09:05:01 Processing... errors/notspam with 1 files
Jan-28-16 09:05:01 ignore and remove files older than Sep-11-88 10:05:01
in folder errors/notspam
Jan-28-16 09:05:01 Imported Files for HeloBlackList: 0
Jan-28-16 09:05:01 Imported Files for Bayes/HMM: 0
Jan-28-16 09:05:01 Finished in 1 second(s)
Jan-28-16 09:05:01 info: corpusnorm after processing errors/spam and
errors/notspam is Spam Weight: 8280 / Not-Spam Weight: 0 => norm: 10.000
Jan-28-16 09:05:01 info: require approx. 6,726 files (3,255,584 words)
from folder spam to get the wanted corpusnorm (1.000)
Jan-28-16 09:05:01 /data/assp/spam
Jan-28-16 09:05:01 File Count: 11,195
Jan-28-16 09:05:01 Processing... spam with 11,195 files
Jan-28-16 09:05:01 ignore and remove files older than Dec-28-15 09:05:01
in folder spam
Jan-28-16 09:15:31 Removed Old: 5
Jan-28-16 09:15:31 Imported Files for HeloBlackList: 11,190
Jan-28-16 09:15:31 Imported Files for Bayes/HMM: 6,672
Jan-28-16 09:15:31 Finished in 630 second(s)
Jan-28-16 09:15:31 info: require approx. all files (3,264,527 words)
from folder notspam to get the wanted corpusnorm (1.000)
Jan-28-16 09:15:31 /data/assp/notspam
Jan-28-16 09:15:31 File Count: 7,009
Jan-28-16 09:15:31 Processing... notspam with 7,009 files
Jan-28-16 09:15:31 ignore and remove files older than Dec-28-15 09:15:31
in folder notspam
Jan-28-16 09:25:53 Removed Old: 7
Jan-28-16 09:25:53 Imported Files for HeloBlackList: 7,002
Jan-28-16 09:25:53 Imported Files for Bayes/HMM: 6,992
Jan-28-16 09:25:53 Finished in 622 second(s)
Jan-28-16 09:25:53 Generating weighted Bayesian tuplets
Jan-28-16 09:26:10 populating Spamdb 503166 records - Bayesian check is
now disabled
Jan-28-16 09:26:23 done - populating Spamdb records - 503166 - Bayesian
check is now enabled
Jan-28-16 09:26:23 done - Generating weighted Bayesian tuplets
Jan-28-16 09:26:23 Bayesian Pairs: 503,166 now in list
Jan-28-16 09:26:23 Generating consolidated Hidden-Markov-Model database
from 3,772,337 record model
Jan-28-16 09:28:22 HMM sequences: 1,848,357 now in list
Jan-28-16 09:28:22 generating Spamdb.helo records from 7,502 collected
HELO's
Jan-28-16 09:28:22 cleaning old Spamdb.helo records
Jan-28-16 09:28:22 done - cleaning old Spamdb.helo records
Jan-28-16 09:28:22 HELO Blacklist: 4 new, 0 now in list
Jan-28-16 09:28:22 Spam Weight : 3,264,527
Jan-28-16 09:28:22 Not-Spam Weight: 3,265,258
Jan-28-16 09:28:22 Corpus norm: 0.9998 - (very good - balanced)
Jan-28-16 09:28:22 Corpus confidence: 0.06250000
Jan-28-16 09:28:27 Start populating Hidden Markov Model. HMM-check is
disabled for this time!
Jan-28-16 09:28:27 start populating Hidden Markov Model with 1,848,357
records!
Jan-28-16 09:28:59 Finished populating Hidden Markov Model with
1,848,357 records!
Jan-28-16 09:28:59 Finished populating Hidden Markov Model. HMM-check is
now enabled again!
Jan-28-16 09:28:59 Total processing time: 1,439 second(s)
Jan-28-16 09:28:59 Total processing data: 118.85 MByte
Jan-28-16 09:28:59 Rebuild processed 14.52 files per second.
Jan-28-16 09:28:59 After finishing the Rebuild process, the
/data/assp/tmpDB folder contains 791.45 MByte.
Jan-28-16 09:28:59 After finishing the Rebuild process, the drive that
contains the /data/assp/tmpDB folder has 1.22 GByte free space from
total 1.90 GByte.
Jan-28-16 09:28:59 building new GripList records and bounce report
Jan-28-16 09:28:59 processing Logfile /data/assp/logs/maillog.txt
Jan-28-16 09:28:59 processing Logfile /data/assp/logs/16-01-27.maillog.txt
Jan-28-16 09:29:01 processing Logfile /data/assp/logs/16-01-26.maillog.txt
Jan-28-16 09:29:02 processing Logfile /data/assp/logs/16-01-25.maillog.txt
Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-24.maillog.txt
Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-23.maillog.txt
Jan-28-16 09:29:03 skipping bounce report because 'DoNotCollectBounces'
is switched ON
Jan-28-16 09:29:03 Uploading Griplist via Direct Connection
Jan-28-16 09:29:04 Submitted 6,924 bytes: 0 IPv6 addresses, 768 IPv4
addresses
Jan-28-16 09:29:04 Trashlist was saved to /data/assp/trashlist.db
--
Dossy Shiobara | "He realized the fastest way to change
do...@panoptic.com | is to laugh at your own folly -- then you
http://panoptic.com/ | can let go and quickly move on." (p. 70)
* WordPress * jQuery * MySQL * Security * Business Continuity *
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally
privileged and protected in law and are intended solely for the use of the
individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no
known virus in this email!
*******************************************************
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
