On Wed, 11 Apr 2001, Tod Thomas wrote:
> I was under the assumption that htdig would follow any links within
> the confines of the 'start_url' and 'limit_urls_to' tags so I was
> expecting that the spider would follow the link to the new server and
Yes, this is correct.
> index its contents too. It doesn't seem to be doing that. Is there
> something I need to add (or remove) from the config file to make this
> work? I'm using htdig V3.1.5.
My first guess is that you are running with "update" digs and your
webserver is not sending out proper Last-Modified headers. In this case,
the date would get set to "0" for both the previous version and the
revised version and htdig will assume them to be unmodified pages.
To fix this, set this in your config file:
modification_time_is_now: true
--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
