I vaguely expect something like this: * All incoming requests go into a pending queue. * Requests in this queue are handed off to warm instances only. * Requests in the pending queue are only sent to warmed up instances. * New instances can be started up based on (adjustable) depth of the pending queue. * If there aren't enough instances to serve load, the pending queue will back up until more instances come online.
Isn't this fairly close to the way appengine works? What puzzles me is why requests would ever be removed from the pending queue and sent to a cold instance. Even in Pythonland, 5-10s startup times are common. Seems like the request is almost certainly better off waiting in the queue. Jeff On Mon, Jul 16, 2012 at 12:00 AM, Richard Watson <[email protected]> wrote: > > I don't get this, yet. A request is coming in. App Engine knows there is > an idle instance with the exact purpose of "absorbing traffic", and a > request would obviously count as "traffic". If we don't know how long the > warmup request would take, surely that applies to the user request as well? > So which is worse - fulfilling the user request with an instance we know > will work while handling the vagaries of the warmup out-of-band, or shooting > the customer request directly at said vague warmup process, leaving a > pristine idle instance to handle other traffic that we aren't guaranteed > might come? > > Maybe the definition of "idle" shouldn't be as binary as it is. An instance > that is doing 10% of it's potential is pretty close to idle. I'd rather only > warm up a new instance when the first one hits, e.g. 70%, rather than always > paying for a just-in-case instance and thereby guaranteeing that I'm paying > too much. Having said that, I have no experience with hugely spiky traffic > so maybe others know why this idea sucks. But for users who measure their > instance count at less than 10, surely this would work better? > > Also, surely each time a warmup is performed App Engine could just add the > startup time to its knowledge? "This app starts within X seconds 90% of the > time." > > -- > Richard > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/iKK_n1yAX8EJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
