http://www.freefind.com/library/howto/robots/
The robots.txt is really to control how your page is indexed by search engines. This file should not be used for access control nor security, and you can't even guarantee that search engines obey the file. Most respectable spider's follow that standard though. You should disallow folders which you don't want to be indexed by search engines and show up in web searches. ~Brad -----Original Message----- From: Ken Ketsdever [mailto:[EMAIL PROTECTED] Sent: Monday, January 09, 2006 3:32 PM To: CF-Talk Subject: Robots.txt - - best practices When do you use a robots.txt file? What directories do you disallow? Is marking a directory as disallowed just a roadmap for hackers? I have a directory of XML data. That data is only accessed by CF, then served up. Should I disallow access to that directory? There are no links to the directory so a robot shouldn't be able to find it anyway? Should they? Confidentiality Notice: This message including any attachments is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender and delete any copies of this message. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Message: http://www.houseoffusion.com/lists.cfm/link=i:4:228924 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
