robots.txt is intended as a mechanism that directs the good guys on how to index your site (the search engines that want to follow the rules). Personally, I wouldn't use robots.txt as a way to hide a directory, although you still might want to tell the SEs to ignore it.
> -----Original Message----- > From: Ken Ketsdever [mailto:[EMAIL PROTECTED] > Sent: Monday, January 09, 2006 2:32 PM > To: CF-Talk > Subject: Robots.txt - - best practices > > When do you use a robots.txt file? > > What directories do you disallow? > > Is marking a directory as disallowed just a roadmap for hackers? > > I have a directory of XML data. That data is only accessed > by CF, then > served up. Should I disallow access to that directory? There are no > links to the directory so a robot shouldn't be able to find it anyway? > Should they? ---------------- [INFO] -- Access Manager: This transmission may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. A2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Message: http://www.houseoffusion.com/lists.cfm/link=i:4:228923 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
