Oooooooo, not sure that's gonna be the best solution. Firstly, if you let Google crawl the secure content, I'll go to Google and view the cached version of your content.
Secondly, the way to detect if it's Google can be spoofed. It'll make itself known to you via the user agent. Dump the CGI scope to find that. Thirdly, I reckon, and I'll need someone else to confirm this, that if you let Google in to index content that's not available to a non member and Google finds out, it'll penalise you. Adrian Building a database of ColdFusion errors at http://cferror.org/ -----Original Message----- From: Doug Boude (rhymes with 'loud') Sent: 12 October 2008 22:20 To: cf-talk Subject: How does Security affect search engine spiders? Hi all. I am curious if anybody knows how securing a site affects a search engine spider's ability to crawl it. For instance, if I have my entire site secured by means of authentication so that any page request is redirected to the login page if the appropriate security creds are not present in session, do spiders receive the same treatment? Are they also prohibited by my security from crawling any page except the login page? If this is true, what can I do to allow spiders to have access to crawl content but still apply security to regular "human" visitors? My only thought on that is to detect the fact that they are a spider (not sure how to do that though) and not implement security in that case. Thanks for your ideas and thoughts. Feel free to email them to me at [EMAIL PROTECTED] Doug :0) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;207172674;29440083;f Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:313790 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
