On 5/7/05, Thomas Chiverton <[EMAIL PROTECTED]> wrote: > Maybe this thing will honour robots.txt
Maybe someday but as of now it does not. Also regarding the comment about the Google spidering capabilities, a spider can't get in behind authentication. This thing can. Yeah sure you're supposed to use a form POST for all potentially destructive code. thing is the real world doesn't work that way. In spades. I think the comments section of that blog explain this reality far better than I can or should, other than to say the rules wouldn't be so commonly broken if there wasn't a reason to do so. -- --mattRobertson-- Janitor, MSB Web Systems mysecretbase.com ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:205967 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
