>>I'd like to pause the htdig crawl and restart it where it left off, is
>>there any way to do this ?
>
>Sure. Pass the -l flag when you call htdig and it will write out the
>"todo" list of URLs before it quits. Then when you restart with this
>flag, it will read this list in and resume.
thanks - I guess I had missed it in the docs ... Which btw don't mention
what signal is to be sent to interrupt : a simple kill or a kill -s INT ?
Next question : how do I know when the crawl has finished to do merge etc
?
My idea :
repeat forever
do
start crawl at 1.15 am (using a work db)
pause crawl at 6.15 am
until done
merge databases
end repeat
thanks a bunch
Franck Horlaville,
Technical Director - Athena Online s.a.
Web site creation and hosting
--
<http://www.athena.online.co.ma/>
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
