Jonathan Knoll: >User-agent: aaaaaa >Disallow: /cgi-bin >Disallow: /stuff >Disallow: /xxxxx.html [...] >User-agent: * >Disallow: / >Wouldn't this just disallow everyone from everything?
No, the file is perfectly OK... The "*" has a special meaning in the Standard: "every other User-agent (not mentioned yet)". So here aaaaaa to ddddddd have limited access, and all >other< bots are banned. Cheers, Tuomas
