Hi Eric, I'm not sure if you've seen the documentation on app caching, but it seems like this is related to the questions you have about the initial load time:
http://code.google.com/appengine/docs/python/runtime.html#App_Caching Of the five ideas that you listed, all of them would probably have some positive improvement but I'd personally lean towards 3 and/or 5 (profiling + inline imports and wait for improvements). I think these would be the least disruptive to the code you've already written and, as you mention, the initial startup cost should effect a smaller percentage of requests as your app sees more usage. It is really up to you though. Your idea about speeding up access to commonly accessed files is interesting, and I think we may already do something similar (also described under app cachine). I'm interested in hearing more, please elaborate :-) Happy coding, Jeff On Apr 14, 7:29 pm, Eric Walker <[email protected]> wrote: > I apologize for the recent spate of posts. I don't think I'll need > much attention once I get going. This is partly a Django-specific > question and partly a general question, so I'll ask it here rather > than on the python list. > > I'm seeing latencies of 2000 to almost 3000 milliseconds with a small > GAE application in connection with requests that are reading a short > list from memcache. I don't think times like these are going to work > for me, and I'm trying to figure out how to proceed from here. > > I'm using Django 1.0.2 together with app-engine-patch 1.0. There's a > possibility that I'm doing something wrong, but after reading a number > of posts, the impression I get is that this is not something that is > out of the usual for this setup. The problem seems to be due to a > zimporter call that is made to load Django. Logs that don't include > this call are as low as 800 ms; not an ideal time, but one I think I > can live with. > > I understand from what I've read that the high latency is due to the > entirety of the application image being loaded with each request, > either because the request hits a different server or because a server > that previously served a request has flushed the application from > memory in the intervening time. If I've understood things correctly, > the application would be loaded into memory on certain of the servers > and response times would go down if there were to be a sustained > increase in traffic. But since the traffic to my application is very > low right now, this doesn't happen. > > Following are some possible ways forward, and I'm wondering whether > anyone has any thoughts on which of them might be the best way to go: > > 1. Back-port the application to Django 0.96, which I understand is > memory-resident on the GAE servers, on the assumption that things > would significantly speed up given my application's traffic profile. > I don't think I've used too many Django 1.0 features at this point, so > hopefully this wouldn't be difficult. > > 2. Stick with the current setup, which I like in other respects, and > remove the unused portions of the Django sources, as is suggested > inhttp://code.google.com/appengine/articles/django10_zipimport.html. > > 3. Stick with the current setup, profile the application carefully and > do things like inline import statements into application code and use > Django templates more carefully, on the assumption that the problem > arises because I'm not using Django efficiently rather than as a > result of the zimport call that is being made. > > 4. Port the application over to a lighter-weight framework. > > 5. Live with 2000+ ms latencies for now, since the App Engine > infrastructure will probably handle a case like this more optimally in > the medium term. > > In a different connection, one question I had for the Google folks as > I was thinking through this issue was whether it might not make sense > to keep a cryptographically secure hash of each file for which a > system call is made (e.g., in connection with an import statement in > python). When a given hash is seen a lot, the file could be pushed > out onto the servers into read-only memory and a copy-on-write done as > necessary. This was just a thought. There might be some securityissuesthat > would prevent such an approach. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
