Re: [ogf-d20-l] RE: NPO (was RE: Domain Abuse )

[Mike Looney]

[EMAIL PROTECTED] wrote:

> I don't think that is what he was implying.  By saying 'no robots.txt'
> he means 'we aren't paying any attention to getting traffic'.  Why
> in the world would you purposely avoid traffic?
>

There are parts of peoples web sites that you rather people not be able to jump
straight to via a search engine.   If you have a page that is linked and is not
covered in your robots.txt file, you _will_ get direct from search engine hits,
with out going through what ever front end you thought they were going through
first.   This could be bad for company, for example.