i think you have to put it just on the root directory u can see this page for help
http://www.searchtools.com/robots/robots-txt.html --- [EMAIL PROTECTED] wrote: > where do i put robots.txt file(s)? in every url > directory i want to search? > (i cannot put it in apache, et al). > > while i'm at it if anyone can point toward > instructions on exactly > how to make one... i be grateful. > > (i have very weird luck with searching on the htdig > site?) > > TR > > p.s. i promise i'm about to stop asking so many > questions that i > ought to be able to find sooner (or later) than i > can. > > _______________________________________________ > htdig-general mailing list > <[EMAIL PROTECTED]> > To unsubscribe, send a message to > <[EMAIL PROTECTED]> with a > subject of unsubscribe > FAQ: http://htdig.sourceforge.net/FAQ.html ===== Jose Julian Buda WebMaster - www.noticiasargentinas.com Noticias Argentinas "So�e con un mundo , un mundo sin Windows..." __________________________________________________ Do You Yahoo!? Buy the perfect holiday gifts at Yahoo! Shopping. http://shopping.yahoo.com _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
