This is just a not that I solved this issue for me. Because of a bug in my code in two places the URL Fetch was creating a near infinite loop and that kicked on the throttling... It was always kicked off by a google bot to start because the page with the problem only existed out in the land of things that used to exist, not in any page a user could get to.
The moral of the story is too fold.. Consider not taking URLFetch's default to follow redirects, and consider blocking requests from APPEngine IP address ranges to prevent loop backs. The Good news... Since I fixed this issue GAE has been amazing and taken 95% of the load off of my wordpress install, and is serving pages in under 1 second unless they have a big picture, but typically pages serve off of GAE faster than Wordpress can serve the HTML portion of the page. http://www.google.com/enterprise/marketplace/viewListing?productListingId=77 71+15478775636094085578 -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Brandon Wirtz Sent: Tuesday, January 18, 2011 3:35 PM To: [email protected] Subject: RE: [google-appengine] Re: throttle_code=2 My app is basically just an Reverse Caching Proxy When the problem starts I get things like 72.14.194.49 - - [18/Jan/2011:15:29:17 -0800] "GET /2010/12/product-reviews/site-reviews/is-your-college-bookstore-making-you-f eel-used-rent-textbooks-online-rather-than-on-a-college-campus/ HTTP/1.1" 500 124 - "AppEngine-Google; (+http://code.google.com/appengine; appid: drakaalfirst),gzip(gfe),gzip(gfe),gzip(gfe)" "www.xyhd.tv" ms=19012 cpu_ms=23 api_cpu_ms=0 cpm_usd=0.000720 pending_ms=9152 throttle_code=2 In my case it looks like IP 72.14.194.49 a Google Bot came through made 3680 requests in 33 minutes and triggered the throttling. When that happens it seems other people randomly get throttled too. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of mathijs Sent: Tuesday, January 18, 2011 3:41 AM To: Google App Engine Subject: [google-appengine] Re: throttle_code=2 We are seeing this to, and I don't know exactly what it means either. Occasionally it seems to mean that the average latency of requests is too high, and that's why more instances are not being spun up. I opened this request: http://code.google.com/p/googleappengine/issues/detail?id=4405 and as I mention there, it sometimes also happens when the request latency seems just fine, so it's unclear why instance spinup is being throttled. On Jan 18, 8:48 am, "Brandon Wirtz" <[email protected]> wrote: > Tonight I'm getting a LOT of " throttle_code=2" in my log, and > whenever that happens URL Fetch seems to stop working. > > How do I avoid these weird throttles? Since they seem to cause my > users to not be served anything, which makes me sad. > > -Brandon -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
