where do i put robots.txt file(s)? in every url directory i want to search? (i cannot put it in apache, et al).
while i'm at it if anyone can point toward instructions on exactly how to make one... i be grateful. (i have very weird luck with searching on the htdig site?) TR p.s. i promise i'm about to stop asking so many questions that i ought to be able to find sooner (or later) than i can. _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
