Mark Roberts wrote:
> "David S." <[EMAIL PROTECTED]> wrote:
> >
> >Could you explain about the "robots.txt"?
>
> It's a plain text file kept in the root directory of your web site which tells
> search engine spiders and other robots which directorys to avoid.
>
> Mine looks like this:
> User-agent: *
> Disallow: /temp3/
> Disallow: /temp2/
> Disallow: /temp1/
> Disallow: /temp/
> Disallow: /graphics/
> Disallow: /run/graphics/
> Disallow: /dfmc/images/
> Disallow: /dfmc/thumbnails/
>
> All the major search engine will obey robots.txt
> Spam bots that harvest email addresses will not.
>
Thanks for the info. I will try to remember that when I get my web site running.
David S.
-
This message is from the Pentax-Discuss Mail List. To unsubscribe,
go to http://www.pdml.net and follow the directions. Don't forget to
visit the Pentax Users' Gallery at http://pug.komkon.org .
