FL wrote:
>
> Hi!
>
> To index my quite huge database, I decided to run a loop :
>
> ./index -N5 -R5 -n 5000 2>log
> ./index -D
First, this is wrong, you should instead set up something like
while /bin/true; do
index -N5 -R5 2>log &
sleep 50
index -E
index -D
done
_OR_
while /bin/true; do
index -N5 -R5 -n 5000 2>log
index -D
done
Second, if you have 400 sites, use more threads, like -N20 or even -N50.
> The program is lauched each hour via cron.
>
> I have noticed that the robot.txt files seems to be always downloaded
> before real indexing starts, wich is not quite efficient with my
> configuration (with about 400 differents Server commands).
>
> What -q tag really means ? Will it solve my problem ?
--
[EMAIL PROTECTED] ICQ 7551596 Phone +7 903 6722750
Hard work may not kill you, but why take chances?
--
