>>in case it's a search spider, you can try adding appropriate robots.txt
robots.txt file may be useful for well behaved robots like Google or Yahoo, but bad bot disguise themselves like "Explorer" or "Mozilla" and they do not comply to robots.txt directives. Most of them never read it, and some of them will even read it ... and visit the forbiden areas on purpose. This is one of the trick I use to detect them : attempt to read pages referred in robots.txt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:327291 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
