I think this is key. Without understanding why thousands of entities are needed on a page, it is hard to offer optimization suggestions. Although the ones offered so far certainly are good to consider - but the _real_ optimization will probably come through a slight (or perhaps significant) redesign.
One other thing about list of keys ... if you can't maintain an actual list that you need to get, you might be able to build some semantics into your keys such that you can construct them. For example, if you need a contiguous range and you know the total range, on create, you could build such info into the key and use that knowledge at fetch time. On May 10, 5:22 am, Greg <[email protected]> wrote: > I'd take a step back and look at WHY you need to fetch these massive > datasets - are you really displaying every single one of thousands of > records on one page? If you are, you users would thank you for > redesigning to a summary/drill-down pattern. > > If you're aggregating the data in some way, then store/cache the > aggregate. For instance, if you were graphing all this data, you could > cache the image and simply add a new line of pixels to it for new > data. > > Another trick is to exploit get_by_key(), which optionally takes a > list and is much faster than all().filter(). So if you can maintain a > list of keys, this will get your objects faster. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
