Kir Kolyshkin
Tue, 24 Sep 2002 00:01:55 -0700
First, this should go to aseek-users mailing list, because you are not sending a patch, asking for a feature or found a bug in ASPseek.
Second, this is not a bug, because when the program finishes, operating system (being it a latest linux kernel or another OS) releases all the memory it used, whether the program freed it or not (the only exception is SysV shared memory, which ASPseek do not use). Third, when reporting about such incident, only words are not enough - numbers are needed badly, too. E.g. output of 'free', 'vmstat', 'top' during and after executing a program would be very nice to have. OK, back to the point. First, some memory is eaten by MySQL and it is not released because MySQL is still running. You can tune the max. amount of memory MySQL uses via it's config file. Second. All the other memory not occupied by programs is occupied by linux kernel buffers (mostly disk cache) - it is freed only if more memory is needed by programs. Basically Linux kernel tries to have very few free memory - because free memory means a waste of resource. Third. System reboot is a bad habit that comes from Windoze experience; this is not a cure for a Linux box. In short - never reboot the box, try to find out what's happening - most of the times this is either not a problem (in your case), or you can fix the problem without a reboot. The only problem you can't fix is linux kernel that screwed up - this is rarely a case, but if this is the case, you should switch to some other kernel - which obviously requires a reboot :) PS a little swap usage is not an indicator of problem and should be considered as normal. In fact, for older 2.4.x kernels (where x is less than 9 or 11 - I do not remember exactly), swap usage less than or equal to RAM size is OK and does not indicate that you need to install more RAM. J and T wrote: > Today I started with an empty index and inserted 175,000 URLs: > > ./index -i -f myurls.txt > > Then I ran the indexer: > > ./index -N 100 -s 0 > > Consumed a bunch of memory once all was resolved and running. In fact > I'm running 2GB DDR 200MHZ RAM on a dual 2.2GHZ Zeon box with Linux > running the latest kernel. While running index it will not only consume > the entire 2GB, but also about 2,000k of disk swap. Once index is > complete none of the RAM was released. The only way I can get it back so > other programs were not swaping memory to disk is to do a reboot. This > process took 7 hours by the way. > > Has anyone experienced the same problem? I could easily index 1,000,000 > URLs all at once using mysql and a multi-threaded perl indexer with > minimum RAM consumption and when done (about 20 hours later) all RAM is > released. Maybe it's because I wrote the Perl script? All other programs > I have used release the RAM back, but not aspseek's indexer for some > reason. -- -- [EMAIL PROTECTED] 7551596@ICQ [EMAIL PROTECTED]