At 2:43 AM -0500 12/1/01, [EMAIL PROTECTED] wrote: >where do i put robots.txt file(s)? in every url directory i want to search? >(i cannot put it in apache, et al).
No, it goes in the root of the server, e.g.: http://www.foo.com/robots.txt >while i'm at it if anyone can point toward instructions on exactly >how to make one... i be grateful. http://info.webcrawler.com/mak/projects/robots/norobots.html >(i have very weird luck with searching on the htdig site?) Define "weird." -- -- -Geoff Hutchison Williams Students Online http://wso.williams.edu/ _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
