> -----Original Message-----
> From: Orjan Sandland [SMTP:[EMAIL PROTECTED]]
> Sent: Friday, March 23, 2001 9:17 AM
> To: [EMAIL PROTECTED]
> Subject: Webboard: Premature end of indexing?
>
> Author: Orjan Sandland
> Email: [EMAIL PROTECTED]
> Message:
> I'm running latest mnogosearch and redhat 7 with mysql rpm installed.
>
Redhat7.
Grrrr.
RPM.
Grrrr
OK, I'll put my pettiness aside.
> I compiled mnogosearch with support for pthreads btw.
>
Good.
> Two days ago, I had about 200.000 urls indexed, with 125.000 of them being
> status OK, about 20.000 was status not modified.
> This morning, after running the indexer all night, the total count is up
> to 255.000 urls, but the Gateway Timeout count was over 150.000, and only
> 60.000 had OK status.
>
Theory: there was an outage, and max_retry_errors [or whatever it's called
in the config file]was reached. the indexer then gave up.
> Yesterday I started the indexer to reindex, using -a option.
> Why did so many get into status Gateway Timeout??
> My first thought was that there was some huge network error (most of the
> pages I index are separated from the server by the atlantic ocean :-).
>
> I can live with this (as long as they get indexed again at some point ;-),
> but I tried to force it to index the urls with Gateway Timeout. Running
> ./indexer -s 504 only works for 4 seconds, then gives me the "Done"
> message.
>
"indexer -s 504" would only have re-indexed out-of-date documents.
If, the night before, it had indexed all of the but given timeouts, they
would be marked in the database as indexed, and have to wait for the default
of 2 weeks before they get indexed again.
You should have hit it with a "indexer -a -s 504". That would have reindexed
all the 504s.
> Am I doing anything wrong? I'm quite new with this, still learning.
>
Naaah. Looks like you know what you're doing [although linux isn't the best
choice from my experience - I use solaris]
> I'm realising that I will need to learn alot, because at this point, with
> 250.000 urls, I've only partially indexed 200 of the 1-2000 websites I
> intend to index....
>
Hope this helps,
Gary (-;
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
