Thanks also by using: User-agent: * Disallow: / That would allow all BOTs to hit your site but not index any pages. Is that right? so in theory it would index just the URL and the META tag data?
>>> [EMAIL PROTECTED] 07/17/03 08:49AM >>> oi Randell!! http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=how+to+build+a+robots.txt&btnG=Google+Search first return looks like it might splain it... mine all just say: User-agent: * Disallow: / :) ------------------------------------ Thursday, July 17, 2003, 8:43:30 AM, you wrote: RBA> Does anyone have a good resource for building a robot.txt RBA> file for search engine bots? RBA> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=5 Subscription: http://www.houseoffusion.com/cf_lists/index.cfm?method=subscribe&forumid=5 This list and all House of Fusion resources hosted by CFHosting.com. The place for dependable ColdFusion Hosting. http://www.cfhosting.com Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.5
