> that I get when I turn the debugging on. Is there any way in htdig to
> overwrite this ?
No. That would violate the robot exclusion standard.
> Robots.txt line: User-agent: *
> Found 'user-agent' line: *
> Robots.txt line: Disallow: /
This tells all robots to ignore all files on this server. So ht://Dig is
doing exactly what it's told.
My suggestion would be to have the server administrator change the
robots.txt file.
--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word unsubscribe in
the SUBJECT of the message.
