So: I was looking at a robots.txt file and it had a series of disallow instructions for various user agents, and then at the bottom was a full disallow: ----------------------------------------- User-agent: aaaaaa Disallow: /cgi-bin Disallow: /stuff Disallow: /xxxxx.html
User-agent: bbbbbb Disallow: /cgi-bin Disallow: /stuff Disallow: /xxxxx.html User-agent: cccccc Disallow: /cgi-bin Disallow: /stuff Disallow: /xxxxx.html User-agent: dddddd Disallow: /cgi-bin Disallow: /stuff Disallow: /xxxxx.html User-agent: * Disallow: / ------------------------------------------ Wouldn't this just disallow everyone from everything? Jonathan S. Knoll Catalog Analyst Aeneid Corporation - provider of EoCenter Try EoCenter at http://www.eocenter.com 282 Second Street, 4th Floor San Francisco, CA 94105 415. 538.8555 ext. 284
