At 9:46 AM -0700 6/12/00, Kevin Obbayi wrote:
>1. The directory structure is also being indexed, so that after I index a
>number of files that I have in the http://www.foo.com/mydir when I type in a
>search query like the word "index" it returns the following link "index
>of/mydir" and then when I click on this link I get the directory listing of
>that directory. How can I prevent this?
Sure. You probably want to put a META robots tag in the header of
your Apache directory index files.
<meta name="robots" content="follow,noindex">
>2. IF I have one "index.html" file in the directory, rundig will not index
>any other files that directory.
Well htdig only follows links. So if you make an index.html file and
it doesn't have links to anything then you won't see anything
indexed. If the server generates an on-the-fly index, it will have
links to the individual files.
--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
