Thanks for the reply! On Saturday, January 24, 2015 09:08:11 AM Thomas Eckardt wrote: > >I have been running ASSP for 3809 days on Gentoo Linux. > > I don't think that V2 will run on this system, if it is really over 10 > years old!!!
Who said the system was 10 years old? The ASSP installation, base files, spam db, etc has been migrated from different servers several times for that period of time. Its running on an up to date install of Gentoo Linux. Not a system that old, just the stats go back that far. > First check your system design. > - at least 4 CPU cores > - at least (absolute minimum) 2GB RAM on 32Bit Perl / 4GB on 64Bit Perl That is a bit excessive given the length of time I have ran ASSP for. Presently its on a dual core VM with 4GB of ram. Which I feel is a bit overkill, about twice the needed resources based on historic usage. > - use at least Perl 5.16.3 - the best unicode processing you'll get with > Perl 5.20.1 /usr/lib64/perl5/5.18.2 I can upgrade to 5.20.1 if recommended. 5.18.2 is the present stable version on Gentoo not that it means anything about being stable or not. I run 5.20.1 on other systems, and many times unstable stuff can be more stable on Gentoo. Newer versions of software, bug fixes, etc. > - install possibly required software (OpenSSL is required) - this depends > on which Plugins are used (ASSP_DCC: dcc, - ASSP_OCR: ImageMagick, > tesseract, clobber/getpdftext/pdftk ) I went through and made sure most all modules where there. Many more than I believe I had even with past installs. > - install a database engine (BerkeleyDB, MySQL, Postgres, DB2, Oracle, > MSSQL .... ) - you must be able to maintain this database (repair, backup, > restore, configure) I have always just gone with BerkleyDB, standard Linux db for such things. > - install and configure ClamAV and/or if you want another Virus-scanner I haven't rant that except maybe the first year I ran ASSP. Its is a waste IMHO not necessary or needed. I have zero problems with viruses and the like for a considerable time. That tends to be just as spammy as other non-malicious spam. > - make sure, you have at least two perfect configured and fast DNS-servers Always have, though in different geographical locations, one local one remote. > - some modules (eg. Sys-MemInfo, Crypt-GOST) available on CPAN could not > be compiled on some systems - download and compile the version provided in > the 'ASSP V2 module installation' folder on SF They are all compiled, you compile most everything on Gentoo. > - run the assp module installer No, modules must be installed and maintained by the system not ASSP. > - bring all Perl modules up to date - Routinely done along with system updates weekly for over a decade. > - start assp and check the startuplog for warnings and errors None, that is 101 stuff > - check the file assp/moduleLoadErrors.txt for errors - correct the errors > or disable the modules in the assp config -> module setup section Did that and installed modules I did not have before, mime, cron ones, etc. > - move the main hashes to a database if the startuplog recommends this This I am not familiar with, what are you referring to there? I will have to look into that. > - configure the rebuild process to use a temporary database > (useDB4Rebuild) If that is the option to reduce memory or what ever. I am trying that now. I am unable to rebuild the spam database at all presently. > >I can't use the schedule cron perl modules > > Schedule::Cron is only used by assp to calculate time values from cron > entries. Only the rebuildspamdb and the restart scheduler are using this > module 'really'. > This module leaks no memory nor does it crazy things! It could be other problem in ASSP, but when I was running the schedule cron module I was able to rebuild the spam db without ASSP running away. When I used the schedule cron module ASSP ran away. It maxed out CPU usage, and memory. Which it is doing now without that module, but failing to finish rebuilding the spam database. Regardless I do not see the point in putting the rebuild process under the ASSP process even if it is multi-threaded. It should be its own stand alone process. That can be run by the real cron. Not require perl modules, and not having ASSP do the scheduling, run this task, and have it be part of ASSP memory and other things. If there are problems with rebuilding the spam database as I am experiencing now. It SHOULD NOT effect ASSP itself. Depending on the system layout, if using NFS etc. One could in theory rebuild the spam database using and entirely different machine than ASSP is running on, shaving the same files. That might be a bit extreme, but as an example shows they should be separate of themselves. > Most of the assp code is loaded by all workers on request. Because of this > reason, the memory allocated by assp will typical grow over 1 or 2 days > after it was started. > This is a normal behavior of V2! Memory leak was an incorrect description. I am more concerned with it running away, CPU maxing out for hours across cores. That only happened before when schedule cron was used. Now its happening anytime I try to rebuild the spam database. Along with other odd problems I cannot explain. Now I am unable to rebuild the spam database, when I could before. That could be due to some perl module being updated. > My assp starts at 370 MB memory - and allocaes ~ 580MB < rebuild > 700MB > after running 2 days and stays there. > - 5 SMTP workers > - Perl 5.16.3 - 32 Bit > - using all features and all Plugins > - 11.000 files in the corpus - takes ~ 30 minutes for a complete > rebuildspamdb on a slow system 'Jan-24-15 04:32:38 Rebuild processed 6.85 > files per second.' > - spamdb: 400.000 records > - HMMdb: 1.500.000 records > > - ClamAV at 400MB > - MySQL at 450 MB - holding all main hashes and lists ------------------------------------------------------------------------------ New Year. New Location. New Benefits. New Data Center in Ashburn, VA. GigeNET is offering a free month of service with a new server in Ashburn. Choose from 2 high performing configs, both with 100TB of bandwidth. Higher redundancy.Lower latency.Increased capacity.Completely compliant. http://p.sf.net/sfu/gigenet _______________________________________________ Assp-user mailing list Assp-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-user
