It even overrides the other disalows.
So in this robots.txt nothing is indexed!
Bert
>> Original Message <<
On 23/03/2000, 22:19:57, Jonathan Knoll <[EMAIL PROTECTED]> wrote
regarding Disallows in robots.txt:
> So:
> I was looking at a robots.txt file and it had a series of disallow
> instructions for various user agents, and then at the bottom was a
full
> disallow:
> -
> User-agent: aa
> Disallow: /cgi-bin
> Disallow: /stuff
> Disallow: /x.html
> User-agent: bb
> Disallow: /cgi-bin
> Disallow: /stuff
> Disallow: /x.html
> User-agent: cc
> Disallow: /cgi-bin
> Disallow: /stuff
> Disallow: /x.html
> User-agent: dd
> Disallow: /cgi-bin
> Disallow: /stuff
> Disallow: /x.html
> User-agent: *
> Disallow: /
> --
> Wouldn't this just disallow everyone from everything?
> Jonathan S. Knoll
> Catalog Analyst
> Aeneid Corporation - provider of EoCenter
> Try EoCenter at http://www.eocenter.com
> 282 Second Street, 4th Floor
> San Francisco, CA 94105
> 415. 538.8555 ext. 284
