Hi Ikai, Yes, that was me who turned up at office hours last night. Thanks for the response, both then and now. I've gone ahead and starred the issue you noted; it's an interesting proposal.
I understand the need for loading times, but I have a question as to how this scales. Presumably, the loading times occur as new machines which serve requests are added; normally, I would think that scaling up by only one or two instances at a time is sufficient to meet request per second growth. What if, however, there is a (temporary) exponential growth in the number of requests per second; will that impact the response time of all of those requests? I was going to run some simulations to test this, but if you already have intuition as to what will happen, that would be helpful. To be more clear, if your application can normally be easily serviced by one or two instances (1 to 10 qps), but then shoots up over a very short span of time (less than ten seconds) to 300-500 qps, will all of those requests lag behind while new instances are loaded to service the traffic spike? That is, will the service time for each request be increased by the loading time except for those requests that are able to be served by the original instances? Right now I'm not using any frameworks or any dependencies outside of XML parsing (I had to manually add XML jar files to my war's lib folder in order for XML parsing not to generate an error when deployed, even though it worked fine locally), so I would expect my load times to be modest. The user-facing functionality of the site actually does use a static page with a "loading" message while it queries the server, which I find very useful, but the concern over loading times is generated by the functionality that is invisible to the user. Thanks again for the help. If I find anything interesting out in testing, I'll be sure to post the results online. Regards, - Michael On Mar 4, 11:43 am, "Ikai L (Google)" <[email protected]> wrote: > Michael, > > (Molson from the IRC office hours?) > > Some small percentage of your application's requests will always be > loading requests, as this is us spinning up a new instance of your > application to either grow for capacity or tearing down your instance > and putting it back up as resource allocation demands. We can't > predict when this will happen. You may want to star this issue: > > http://code.google.com/p/googleappengine/issues/detail?id=2456 > > Startup time is generally a function of several different things: > > - Spinning up the JVM (Relatively cheap, but on the order of magnitude > of 500ms - 1s) > - How many dependencies are you loading? (Relatively cheap compared to > JVM spinup) > - Framework init (Can be VERY expensive - loading up a dynamic > language runtime will always take a few seconds. Some frameworks will > also scan every class in your classpath. Spring, for instance, does > this to look for annotations eagerly on init time) > > Strategies to counteract these factors include optimizing for lazy > loading, which spreads the total load time across acess to several > different resources. Not many existing frameworks do this. > > As your application grows, loading requests should account for a > smaller and smaller percentage of your total requests. I've seen > solutions with rich applications that show a static page loading > dynamic resources as a general landing page. This doesn't solve the > load time solution, but it meets the user halfway by making a web app > appear to load faster as opposed to causing a user's brower window to > be blank while waiting for a request to be handled. > > > > > > On Tue, Mar 2, 2010 at 4:32 PM, Michael <[email protected]> wrote: > > Looking at my App Engine logs, I see troubling results when viewing > > the response times for requests. In my current log set, the first 80 > > requests all complete in under 100 ms with less than 100 ms of cpu or > > api time. Then, oddly, the 83rd request, from the exact same client > > with the exact same request parameters, takes 7,192 ms to respond with > > 10,123 cpu ms (and 12 api ms). > > > These kinds of spikes are dotted throughout my logs. They occur in > > less than 1% of cases, as far as I can tell, but the spikes are not > > just large; they're enormous. I know for a fact that the request > > parameters and returned data were identical to the requests several > > seconds before and after from the same client, but the request took > > about 20 times longer to serve. > > > Does anyone know what causes these large spikes in response time, and > > can anyone share tricks to help alleviate these spikes? I know that > > it is somehow related to instantiating the JVM, but I don't know: > > - how to reduce the startup time of the JVM > > - how to predict when GAE will try to start a new JVM > > > Thanks in advance for any advice, > > - Michael > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine-java?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
