>
> I have set up htdig so that every night, it indexes a long list of small
> web sites. In general, this works very well, but I've found that I have to
> be very careful when adding new sites to the end of the list.
>
> Some sites seem to cause htdig to fail. When this happens, htdig doesn't
> continue on with the rest of the list -- it simply skips to the next step
> in rundig. This means that I have to do some careful adding and substracting
> to the list of sites before I figure out what caused the index to fail.
>
> What I would like to do is to somehow index each site separately and have
> some kind of error log if htdig hits a site that it fails on (for whatever
> reason). Then, I would like for it to procede to the next site in the list,
> whether or not it failed to index the previous site.
>
> Thanks for any help,
> Todd Wallace
I do a weekly index of over 900 servers and have never had this happen.
What version of htdig are you using?
Also, I would recommend using the http_proxy attribute in the config
file if you possibly can.
--
David J Adams
<[EMAIL PROTECTED]>
Computing Services
University of Southampton
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>
