Mike, I'm indexing 495.166 pages in total (index -S), so it seems that you have some very large sites on your list.
You can stop the running indexer using "index -E" on another terminal (ssh). Ernesto. On Tue, 29 Oct 2002, Searcher wrote: > >Mike, I'm reindexing 9890 sites a day (those days that everything > >works allright). The first run took much more, but it seems that yours is > >taking too much time to complete. > > Ah, finally, someone who's running a similar task. I can see that the system is > constantly connecting to a good dozen systems or so and pulling down what ever > it's indexing. I know it's alive and I can also see that the drive space is > constantly on the rise so it's not being slow about it that I can tell? > > >Now I'm running on a RedHat 7.3, but before I was using FreeBSD on a > >fairly similar configuration and it was many times slower than now. > > I'm running on RedHat 7.2 myself, with the dual 350Mhz CPU's and 8GB's of memory > as I mentioned. The system seems to be handing everything ok, the load is not > very high or anything but it's been running non stop for over two weeks now. > > >Some times seems that stoping and restarting the indexer is faster than > >simply wait the end of the run. > > I think I started the process from the console and left it there. I've been > monitoring the system from an SSH login. My point is, I can't quit the session > unless I hit CTL-C or is there some other way, from another login? > > >The disk space depends on the content of the pages, but on my case > >it is using 1GB for the cache and 2GB for the database. > > I know it's into the millions of pages because there is constantly a queue of > between 1.2 and 1.5 million pages. It's already into 45GB's of drive space usage > now. I'm indexing 10028 sites, not pages but I notice you also said you are > indexing 9000+ *sites*. I had assumed that the drive usage would be pretty normal > for the number of sites that I'm indexing. > > >By the way, I use the following mysql command to report top 10 sites: > >sometimes there's a huge site that shouldn't be indexed, which is > >consuming lots of resources. > > I'll check this out. > > >Hope some of this may help you. > > Invaluable, thanks very much for your help. > > Mike > > >
