I'd take a step back and look at WHY you need to fetch these massive datasets - are you really displaying every single one of thousands of records on one page? If you are, you users would thank you for redesigning to a summary/drill-down pattern.
If you're aggregating the data in some way, then store/cache the aggregate. For instance, if you were graphing all this data, you could cache the image and simply add a new line of pixels to it for new data. Another trick is to exploit get_by_key(), which optionally takes a list and is much faster than all().filter(). So if you can maintain a list of keys, this will get your objects faster. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
