It even overrides the other disalows. So in this robots.txt nothing is indexed! Bert >>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<
On 23/03/2000, 22:19:57, Jonathan Knoll <[EMAIL PROTECTED]> wrote regarding Disallows in robots.txt: > So: > I was looking at a robots.txt file and it had a series of disallow > instructions for various user agents, and then at the bottom was a full > disallow: > ----------------------------------------- > User-agent: aaaaaa > Disallow: /cgi-bin > Disallow: /stuff > Disallow: /xxxxx.html > User-agent: bbbbbb > Disallow: /cgi-bin > Disallow: /stuff > Disallow: /xxxxx.html > User-agent: cccccc > Disallow: /cgi-bin > Disallow: /stuff > Disallow: /xxxxx.html > User-agent: dddddd > Disallow: /cgi-bin > Disallow: /stuff > Disallow: /xxxxx.html > User-agent: * > Disallow: / > ------------------------------------------ > Wouldn't this just disallow everyone from everything? > Jonathan S. Knoll > Catalog Analyst > Aeneid Corporation - provider of EoCenter > Try EoCenter at http://www.eocenter.com > 282 Second Street, 4th Floor > San Francisco, CA 94105 > 415. 538.8555 ext. 284
