Mike, I'm reindexing 9890 sites a day (those days that everything works
allright). The first run took much more, but it seems that yours is
taking too much time to complete.
Now I'm running on a RedHat 7.3, but before I was using FreeBSD on a
fairly similar configuration and it was many times slower than now.
Some times seems that stoping and restarting the indexer is faster than
simply wait the end of the run.
The disk space depends on the content of the pages, but on my case it is
using 1GB for the cache and 2GB for the database.
By the way, I use the following mysql command to report top 10 sites:
select count(*) qty,site
from urlword,sites
where urlword.site_id=sites.site_id
group by urlword.site_id
order by qty desc
limit 0,10;
sometimes there's a huge site that shouldn't be indexed, which is
consuming lots of resources.
Hope some of this may help you.
Ernesto.
On Tue, 29 Oct 2002, Searcher wrote:
> Ok, so it's been running since I posted this message, no crash, non stop, 24/7.
> It's been indexing for for over two weeks now.
>
> The question remains the same. How long will this indexing take and how much
> drive space will this take using the following;
>
> The bandwidth is a full T1, point to point, the indexing has full use of the T1.
>
> The machine is a dual 350Mhz with 8GB's of memory.
>
> I'm running 'index -N 25'.
>
> I'm running aspseek pretty much out of the box, configured almost as it comes,
> with few changes. This was a trial run but I didn't realise it would take this
> long.
>
> Finally, I am indexing 10028 different sites, not URL's, again, with the basic
> configuration.
>
> The process so far, has used up 45GB's of drive space.
>
>
> And again, I know that no one knows for sure but there must be some users out
> there who have/are run/running a similar indexing process. I'm not looking for
> anything exact, just *some* idea on how much longer this will take and how much
> more drive space it will use up.
>
> Thanks for ANY help you can offer me on this :).
>
> Mike
>
>
>
>
>
> On Fri, 18 Oct 2002 09:07:48 -0700, Karen Barnes wrote:
> >Of course Mike there are many factors involved here. One of the
> >biggest is
> >of course your available resources. If you have a frame relay for
> >example
> >that is maxed at 512kbps and only guaranteed 256kbps, then the speed
> >will be
> >limited to this. I can tell you I have aspseek running on a
>
>
