I don't think the situation is so dire. One option is just to get rid of autoscaling - if that means user requests don't wait for cold starts, I'd consider it an improvement.
Another option is to take the approach of AWS ElasticBeanstalk. The elastic load balancers never send requests to unresponsive servers, and you can configure the system to add or remove instances based on metrics like CPU utilization and request latency. I'm not a big fan of EB (or at least, I wasn't 2 years ago when I last tried it) but the autoscaling system is exactly what I want. Jeff On Fri, Apr 5, 2013 at 12:23 PM, Vinny P <[email protected]> wrote: > > On Friday, April 5, 2013 10:24:26 AM UTC-5, Jeff Schnitzer wrote: >> >> I think what people are looking for is: What combination of settings >> will prevent users from seeing cold starts, or at least decrease the >> probability down to 4 or 5 sigma? There doesn't appear to be an >> answer, not even "run ten thousand resident instances". >> > > Believe me, I sympathize with you, Aswath, Cesium, etc. I'm a big Java on > GAE guy (and so are my clients) and I frequently run head first into this > issue. And of course, Java makes it particularly hard due to its overhead > The only thing we can do is to run continuous testing, repeated inspection > of logs, A/B test adjusting the latency slider and resident instances, etc. > To be fair, I work for a large company alongside a ton of very bright > people; we can afford to spend manpower to continually optimize our > applications. Not everybody can do the same. > > There's just no magic bullet here. As the famous Spolsky quote goes, "all > abstractions are leaky". Is there room to criticize GAE? Yes, obviously, and > I have a huge list of issues I would love fixed (Google, please give us > incoming email/xmpp on custom domains, thanks). But I also have clients that > complain to me about Heroku (especially after they were discovered lying to > users about how routing worked, that day was just nonstop complaining), and > other PAAS. If I had the solution to everything, I'd be selling it to > clients. The only thing that works is continuous monitoring. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
