On Mon, May 7, 2012 at 6:42 PM, Richard Steele <[email protected]> wrote:
> Today we had Google + 3 or 4 other spiders hammering our multi-instance > server at the same time. Is there a way to control these bots to prevent > them from submitting request after request? How do most high traffic > servers handle this? Thanks! > I addition to the advice you've gotten (and please take this as constructive advice) it's possible that if you are having trouble dealing with search engine traffic - you do not, in fact, have a "high traffic server". This may be a good gut-check time to ask yourself "Can we handle increased load if our website becomes suddenly more successful?". Seeing your website hiccup on search crawler traffic may be a good early indicator that you need to do some stress testing and find places to improve. -Cameron -- Cameron Childress -- p: 678.637.5072 im: cameroncf facebook <http://www.facebook.com/cameroncf> | twitter<http://twitter.com/cameronc> | google+ <https://profiles.google.com/u/0/117829379451708140985> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351064 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
