Rest of truncated message, seems list cuts them off? Might have to split into a 3rd....
On Saturday, January 24, 2015 09:08:11 AM Thomas Eckardt wrote: > > - ClamAV at 400MB > - MySQL at 450 MB - holding all main hashes and lists > - Symantec EP 12.1.5 at 1GB Normally ASSP is fine, low memory and CPU usage across the board. I had it running on a single core system for a long time and only dual in recent years. The system hardly required such even when rebuilding the spam database, with NFS root VMs, non-local disks. The bad behavior started when I was using the schedule cron module. Now its happening anytime I try to rebuild the spam database. ASSP maxes out everything on the system. It still continues to function a bit, web interface comes up, but most email related functions slow and some error out, ssl, etc. > >No clue why I need to move a 10yr old cron task to within perl > > via perl cron modules. > > No clue ??? - read the manual -> newReportedInterval I have tried to review the documentation, I will look again specifically at that. > This is because the rebuildspamdb task is a permanent running task in V2. > Both databses are maintened permantly on request (spam/ham reports). > The consolidated rebuildspamdb is required only once a day or even less. That makes sense some what, but then one should still be able to invoke it on its own outside of ASSP. Given that it is a resource consuming process. Also that the larger process does consume such resources, I would be curious about that happening on an ongoing basis. That would seem a bit inefficient with high-volume systems. Though I could see it keeping things more up to date. Does the rebuild task depend on the schedule cron module? If so that is not happening at all in my case without that module. > >Thus I have to log into > > the web interface daily or when ever just to rebuild the spam database. > > It could be done this way, but this is not required. The consolidated > rebuildspamdb task (the same like in V1) could be run scheduled via > configuration. If you want use the system scheduler for any reason - read > the file (distro) assp\cmdqueue_example.txt. The SNMP interface (agentX) > could be also used to control assp from outside. I don't want to control ASSP from outside. I want to use my 10yr old cron task I had setup to rebuild the spam DB on its own at schedule intervals as I had always. I had no issues with that till it was moved into ASSP. I had no issues then till I tried scheduling that via schedule cron from within ASSP. Now I cannot invoke rebuild spam db from the web interface at all without blowing up ASSP. That used to only happen with schedule cron module. Though now its happening when I rebuild the spam db. I think it is happening at other times, but I have not confirmed that yet. > >The bugs and issues with ASSP are getting ridiculous. > > I don't know a big one currently. I'm running V2 for 7 years now - using > the latest version every time. I just switched to it, I was always running 1.x single threaded for 10 or more years. I only did because 1.x is no longer maintained and started crashing perl with SPF code issues. Some change in perl, it effected many other programs it seemed. > >When I do run > > that, it never finishes, maxes out the CPU, cores, etc for hours. > > The rebuildspamdb task is running a single thread only? And yes - this > single thread could use a single core at 100% some times. > Use the 'Worker Status' screen - it will tell you what is going on in the > threads. > To debug the rebuildspamdb task create the empty file > 'assp/rebuilddebug.txt' before starting the task. This file will show > details about this task. I can see ASSP processes extreme amounts of CPU, > 50% for hours, and the system as a whole goes up to over 100% usage of all CPU cores and memory. This last till I kill perl/assp and restart things. I have received alerts about excessive CPU usage lasting more than 2 hours several times. Thus coming to the list about the growing issue it seems. > If you have to process a large corpus, it is a good idea to mount the > 'assp/tmpDB' folder to a RAM-drive. Another solution would be, to used > cached disk IO-system or a SSD. Simply follow the recommendations shown in > the rebuild output (rebuildrun.txt). The VM runs on raid 10 SSDs, and disk IO is better than it ever has been. I had it running on NFS root mounted VM, also on RAID SSD, but the NFS layer added overhead. Even then I never had any issues with rebuilding the spam database, or ASSP running away. I have seen some other odd issues, like a single email account recently started having odd problems sending email. Errors in logs I have never seen before and they got blocked from more than one location and device with different errors. That is unrelated, and a minor issue at the moment. I am more concerned with inability to rebuild the spam database and/or run that process from the web interface without it blowing up ASSP. Which then effects ASSP and all email. I believe that problem has come up on its own and just triggered faster by running the rebuild spam database process from the web interface. I am not sure if there is a problem with the rebuild process as I cannot invoke that outside of ASSP. I know there was a problem before when using schedule cron and I blamed that module. It might not have been that module and have been other things. ASSP has always been a bit buggy. Codebase always been a bit odd. I have never liked it being in perl and remaining there. None the less I have NEVER had major issues like these ever before. Not till the crash with the now unmaintained 1.9.9.x codebase. I moved on, and now have new problems I have never seen before and really dislike. At one point it had all kinds of releases coming out, Fritz was going crazy. The community was putting out its own release a few versions back to bring some sanity with release or something. Keeping up with changes to ASSP and the releases, issues, etc has not been trivial over the years. Though I have slacked in recent times, and need to learn more about the changes to V2 that is for sure. -- William L. Thomson Jr. Obsidian-Studios, Inc. http://www.obsidian-studios.com This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking ------------------------------------------------------------------------------ New Year. New Location. New Benefits. New Data Center in Ashburn, VA. GigeNET is offering a free month of service with a new server in Ashburn. Choose from 2 high performing configs, both with 100TB of bandwidth. Higher redundancy.Lower latency.Increased capacity.Completely compliant. http://p.sf.net/sfu/gigenet _______________________________________________ Assp-user mailing list Assp-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-user
