The robot.txt is a file to control how a web spider , goes though your site
. Not all spiders respect this file.
Try
http://info.webcrawler.com/mak/projects/robots/norobots.html
Justin
----- Original Message -----
From: "Chad Gray" <[EMAIL PROTECTED]>
To: "CF-Talk" <[EMAIL PROTECTED]>
Sent: Monday, October 09, 2000 6:24 PM
Subject: OT: robots.txt
> Anyone know what robots.txt is?
>
> thanks
> --------------------------------------------------------------------------
----
> Archives: http://www.mail-archive.com/[email protected]/
> To Unsubscribe visit
http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk or
send a message to [EMAIL PROTECTED] with 'unsubscribe' in
the body.
>
>
------------------------------------------------------------------------------
Archives: http://www.mail-archive.com/[email protected]/
To Unsubscribe visit
http://www.houseoffusion.com/index.cfm?sidebar=lists&body=lists/cf_talk or send a
message to [EMAIL PROTECTED] with 'unsubscribe' in the body.
