I apologize for the recent spate of posts. I don't think I'll need much attention once I get going. This is partly a Django-specific question and partly a general question, so I'll ask it here rather than on the python list.
I'm seeing latencies of 2000 to almost 3000 milliseconds with a small GAE application in connection with requests that are reading a short list from memcache. I don't think times like these are going to work for me, and I'm trying to figure out how to proceed from here. I'm using Django 1.0.2 together with app-engine-patch 1.0. There's a possibility that I'm doing something wrong, but after reading a number of posts, the impression I get is that this is not something that is out of the usual for this setup. The problem seems to be due to a zimporter call that is made to load Django. Logs that don't include this call are as low as 800 ms; not an ideal time, but one I think I can live with. I understand from what I've read that the high latency is due to the entirety of the application image being loaded with each request, either because the request hits a different server or because a server that previously served a request has flushed the application from memory in the intervening time. If I've understood things correctly, the application would be loaded into memory on certain of the servers and response times would go down if there were to be a sustained increase in traffic. But since the traffic to my application is very low right now, this doesn't happen. Following are some possible ways forward, and I'm wondering whether anyone has any thoughts on which of them might be the best way to go: 1. Back-port the application to Django 0.96, which I understand is memory-resident on the GAE servers, on the assumption that things would significantly speed up given my application's traffic profile. I don't think I've used too many Django 1.0 features at this point, so hopefully this wouldn't be difficult. 2. Stick with the current setup, which I like in other respects, and remove the unused portions of the Django sources, as is suggested in http://code.google.com/appengine/articles/django10_zipimport.html. 3. Stick with the current setup, profile the application carefully and do things like inline import statements into application code and use Django templates more carefully, on the assumption that the problem arises because I'm not using Django efficiently rather than as a result of the zimport call that is being made. 4. Port the application over to a lighter-weight framework. 5. Live with 2000+ ms latencies for now, since the App Engine infrastructure will probably handle a case like this more optimally in the medium term. In a different connection, one question I had for the Google folks as I was thinking through this issue was whether it might not make sense to keep a cryptographically secure hash of each file for which a system call is made (e.g., in connection with an import statement in python). When a given hash is seen a lot, the file could be pushed out onto the servers into read-only memory and a copy-on-write done as necessary. This was just a thought. There might be some security issues that would prevent such an approach. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
