>> Use robots.txt:
>
> This is correct to control good bots, but 
> bad bots don't read it.

This isn't relevant, since the crawlers used by Google and Yahoo do obey 
robots.txt.

> Some (very) bad bots even exploit what 
> is in the file to read what's forbidden.

Robots.txt is not intended to be a security mechanism.

Dave Watts, CTO, Fig Leaf Software

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to 
date
Get the Free Trial
http://ad.doubleclick.net/clk;160198600;22374440;w

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:298917
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Reply via email to