Hi Jon and Ikai, I've been seeing crazy amounts of warmup requests all week. For instance, while one request was being served, 5 to 7 instances started up *simultaneously*, although there was no other load. Now, the *next* request was almost guaranteed to hit one of the new instances, which are warmed up, but not 100% hot... and thus takes a long time to load... which spins up a few *new* instances. The app used to have a startup time of 5 to 6 seconds, and it used to respond within 400ms to 800ms, with only a few heavy pages being a lot slower. Not a lightning fast app, but it worked really well before 1.5.2. Now response times have dropped by factor 3 to 5, which takes even normal requests into a really awkward 2 to 3 second range. This has been plaguing the application all week, and I had no clue what was going on.
I figured out that I needed to limit the the idle VMs to 3 to, so I'd at least avoid having 3-7 new idle VMs per request and always hitting luke-warm instances. That's better, but there is still an awful lot of spinning up and spinning down going on, and every 3rd or 4th request or so still goes to a "warm-but-not-hot" VM, taking 3 seconds or so. I don't really care about the billing at the moment, but it is really embarrassing me in front of our customers to have such a slow application all of a sudden. I've been working the weekend on some general profiling to mitigate, but please make this stop, or give us more control so we can determine how soon a new instance is created (in my case, not so soon...) Does this also change the recommended approach to warmup-request in general? So far, I had only warmed the most crucial parts of the app, since the doc states that requests may as well hit a cold instance, and you wouldn't want to wait for 20s until really the entire app was piping hot. Instead of doing an extreme 20 seconds warm-up, the instances would gradually hotten up with each new request coming in. But now instances go up and down a lot, so each one is barely luke- warm, and thus slower on average. I'd change the warmup sequence to 20s of heavy exercises if it helped, but that would spin up even more instances... oh dear. :) Cheers, Per -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
