Returning a 503 is REALLY REALLY Bad for SEO.
304 Seems to be ignored by Google Bot on GAE, but also Google Bot will try queries to which there are no links, and which no user has ever made. If your latency goes higher Google Bot will throttle back. but the only way to slow down the page serving is to put a wait timer, which burns Instance time. Rob Paul to Pay Peter. From: [email protected] [mailto:[email protected]] On Behalf Of Rajkumar Radhakrishnan Sent: Saturday, September 03, 2011 6:41 PM To: [email protected] Subject: Re: [google-appengine] Re: Google Bot Is Your Enemy @Brandon : This is the case for one of my popular web-sites too. I believe if latency time increases, Google bot will automatically scale down its crawl rate and I also fear that such an increased latency will have a negative effect on the page ranking. By the way, have you started using 304 (Not Modified) responses, for pages which have not modified ? This can reduce the resource usage by Google Bot. Worst case option is to resort to a check for every Nth request (10 < N < 100) from the bot (of course, using memcache) and send a 503 status.. 503 Service Unavailable The server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state. ..and this should also give a hint to the Google bot to scale down its crawl rate. This will be useful when you want to retain better latency and want to hint Google bot alone. Again, this can have an effect on the page rank too. And I am not sure which is worse bad latency or a 503. Anyone else has experience in this space ? Thanks & Regards, Raj On Sun, Sep 4, 2011 at 2:52 AM, Darien Caldwell <[email protected]> wrote: Nice, so basically Google is using their own service to tack on additional charges to your bill. Doesn't sound ethical. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected] <mailto:google-appengine%[email protected]> . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Build online database applications, over Google App Engine. iFreeTools Creator - http://creator.ifreetools.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
