oi Randell!! yeah, I don't even know if they will index the url.... but they go no further than that....(if they use the robots.txt file....)
------------------------------------ Thursday, July 17, 2003, 8:57:55 AM, you wrote: RBA> Thanks also by using: RBA> User-agent: * RBA> Disallow: / RBA> That would allow all BOTs to hit your site but not index any pages. RBA> Is that right? so in theory it would index just the URL and the RBA> META tag data? >>>> [EMAIL PROTECTED] 07/17/03 08:49AM >>> RBA> oi Randell!! RBA> http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=how+to+build+a+robots.txt&btnG=Google+Search RBA> first return looks like it might splain it... RBA> mine all just say: RBA> User-agent: * RBA> Disallow: / RBA> :) RBA> ------------------------------------ RBA> Thursday, July 17, 2003, 8:43:30 AM, you wrote: RBA>> Does anyone have a good resource for building a robot.txt RBA>> file for search engine bots? RBA>> RBA> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=5 Subscription: http://www.houseoffusion.com/cf_lists/index.cfm?method=subscribe&forumid=5 This list and all House of Fusion resources hosted by CFHosting.com. The place for dependable ColdFusion Hosting. http://www.cfhosting.com Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.5
