So, there's a couple of things going on here. I'll see if I can help explain.
The first is that with 1.5.2 we changed how resident instances work, so that if a dynamic instance existed, the scheduler would prefer an idle dynamic instance to an idle resident instance. Further, if a resident instance was busy, we would use this as a cue to warmup a new dynamic instance. The point of these changes is to turn the resident instances into the reserve for the app. They are generally idle, but if the app gets more traffic than it can handle with non-reserved instances, then it will use some of the reserved instances (and this will in turn spawn a new dynamic instance). Generally, Always On is going away with the new billing plan, and being replaced by Min Idle Instances, which is how the reserved instances have been changed to behave with 1.5.2. We're continuing to evaluate all aspects here, both how well these reserve instances are working, what we should be doing, what we should change about the scheduler and the billing plan, and so on. In terms of this specific example, the slow request was caused by general bigtable slowness during that time interval. This can be seen somewhat here: http://code.google.com/status/appengine/detail/datastore/2011/07/27#ae-trust-detail-datastore-get-latency This can also be investigated somewhat using our logs viewer. For example, we can see all loading requests for an app with: https://appengine.google.com/logs?app_id=wordpong&severity_level_override=1&severity_level=3&tz=US%2FPacific&filter=loading_request%3D1&filter_type=regex&date_type=now&date=2011-07-27&time=15%3A46%3A45&limit=20&view=Search. Note how the only loading requests this app has received have been /_ah/warmup Also we can see all requests sent to a specific instance. Here's the one with the log line you listed above: https://appengine.google.com/logs?app_id=wordpong&severity_level_override=1&severity_level=3&tz=US%2FPacific&filter=instance%3D00c61b117c39de5ca2d60c64270fc703fdce1355&filter_type=regex&date_type=now&date=2011-07-27&time=15%3A56%3A40&limit=20&view=Search. Note how the first request the instance served was /_ah/warmup, followed by a pause of 4 seconds, followed by the /game/Game.wp request which ran for 9 seconds. There are a couple of things that can be done now to get different behaviors. One is to set Max Idle Instances to three, which will kill off the dynamic instances for your app, and leave the app with just the resident instances. The other is to use Backends, which will give you direct control over how many instances run for your app and their properties: http://code.google.com/appengine/docs/java/backends/overview.html Hopefully that helps. There is also a lengthy discussion going on at: http://groups.google.com/group/google-appengine/browse_thread/thread/baf439a6e073f6da On Wed, Jul 27, 2011 at 2:59 PM, Mike Lawrence <[email protected]> wrote: > I purchased 3 Always-On instances. > My site is under construction with no traffic. > When I hit my site, GAE fires up a new dynamic instance to service the > request when there are 3 idle instances! > My app starts in 2.0 seconds (using stripes) > But GAE takes 9.4 seconds to reply (why?). > Really annoying. > Why pay for Always-On when you get the crappy response time of dynamic > instances? > > App Id: WordPong > > EST: > 2011-07-27 17:37:04.859 /game/Game.wp?_eventName=questionList 200 > 9481ms 2063cpu_ms 103api_cpu_ms 1kb Mozilla/5 > 2011-07-27 12:37:51 Completed update of a new default version > version=22.2011-07-27T19:37:17Z > > Here's the screen shot of my instances at the time of the request: > http://dl.dropbox.com/u/473572/Untitled.jpg > > Why are there any dynamic instances running at all when there are 3 > idle always-on instances available? > Looks like a serious bug where GAE is wasting resources and providing > poor response times for no reason. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
