|
If you have only 1 host, then it can take long time
to index >100000 URLs.
Possibly problem is that HTTP server gives response
with delay.
Try to run "top" during indexing and see how much is the percentage of CPU time for "index",
"mysqld" and "httpd" (if httpd is run on the same computer)
Alexander.
----- Original Message -----
Sent: Thursday, July 12, 2001 12:53
PM
Subject: Re: [aseek-users] Indexing is
taking a long time
Just 1 host and 1 thread, I cleared out the database, removed
the var/aspseek directory
and started from scratch tonight. this is how I
started it
echo `date` > howlong (save start date and time to a
file)
./index -a
echo `date` >> howlong (save end date and time to
a file)
When I tried starting it over again it wouldn't do more than
1100 documents. Then I went into
the apache conf and turned Indexes on and
now it's running again. I normally shut that off so people
can not list
directories.
I also notice that aspseek has a problem trying to index hrefs
that have mailto: or news: in then such as
< a href="mailto:[EMAIL PROTECTED]">Help</a>, it
calls them unsupported protocol...
DocumentRoot
"/usr/local/apache/htdocs"
<Directory
"/usr/local/apache/htdocs">
Options Includes Indexes
FollowSymLinks
#Options Includes FollowSymLinks
AllowOverride
None
Order allow,deny
Allow from
all
</Directory>
PS:
Very Nice Program, and thanks
for the help.
At 11:40 AM 7/12/2001 +0800, you wrote:
How many
threads (-N parameter) do you run ?
How many unique hosts are you indexing
?
Alexander.
----- Original Message -----
From: "Bob Couch"
<[EMAIL PROTECTED]>
To:
<[EMAIL PROTECTED]>
Sent: Thursday, July 12, 2001 1:32
AM
Subject: [aseek-users] Indexing is taking a long
time
>
> I'm indexing around 170,000 files on a dual PIII
933mhz with 1 gig of ram
> and nothing else running.
> I started
the index 7/10 at 4:30PM EST its now 7/11 and 1:30PM and it's
> still
not done. Why does it
> make several passes through the same documents
over and over again. Is
> something wrong?
>
> PS:
>
I'm running 1.2.4a
>
|
