Try:
<HTML>
<HEAD>
<META NAME="robots" CONTENT="noindex, follow">
</HEAD>
<BODY>
.....
</BODY>
</HTML>
--
David Adams
Computing Services
Southampton University
----- Original Message -----
From: "Miroslaw Prywata" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, August 07, 2001 1:33 PM
Subject: [htdig] htdig and filenames
> Hello,
>
> I use htdig-3.1.5 (RH6.2). I index only local files (several thousands of
> them). They are put in directories and, unfortunately, htdig puts into
> index also those directory names. What is more those names appear to be of
> ,,better quality'' (more stars) than the same words found in standard
> shtml TITLE tag. I created in each directory file that looks like that:
>
> <HTML><HEAD><META NAME="htdig-noindex" ></HEAD><BODY>
> <A href="570001.shtml"></a>
> <A href="570002.shtml"></a>
> .....
> </BODY></HTML>
>
> It should IMHO prevent htdig from indexing the very document but only
> follow links, that are inside. Unfortunately in search dialog I get:
>
> Index of //bla/bla/bla
>
> or
>
> []
>
> with http address of this ,,empty'' page.
>
> Any sugestion? How can I exclude directory names from being indexed? I
> would like to index 10^5 local files located in directories, but only
> those files and nothing more. Now I created links to those directories,
> and that's why they are also indexed, but maybe there's a better way of
> making htdig to do it.
>
>
> Mirek
>
>
> _______________________________________________
> htdig-general mailing list <[EMAIL PROTECTED]>
> To unsubscribe, send a message to
<[EMAIL PROTECTED]> with a subject of unsubscribe
> FAQ: http://htdig.sourceforge.net/FAQ.html
>
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
