Your code for memcache is not correct. The correct code pattern using pseudo
code is the following:
somevar = memcache.somekey
if (somevar is null){
somevar = getitfromthedatastore(...)
memcache.somekey = somevar
}
...
In your code the value for the key you are interested in could be purged
between the time you check for its existence in memcache and the time you
reference it to assign it to a variable.
On Sat, May 8, 2010 at 4:29 PM, Adrian Holovaty <[email protected]> wrote:
> Hi all,
>
> I've got an App Engine application in which I'm selecting hundreds of
> entities from the same model, possibly even 1000 or 2000 of them --
> and I need all of them at the same time, for display in the browser.
> (There is no way around this requirement; I need all of them at the
> same time.) The Python code looks something like this, with model/
> property names changed:
>
> MyModel.all().filter('somekey =', x).order('the_order')
>
> This becomes unbearably slow when selecting hundreds or thousands of
> records. I've read through the docs and various articles explaining
> *why* that happens, but I haven't found any good advice on how to
> solve it, aside from "put it in memcache" or "don't do that." I am
> indeed using memcache in the app's view code, like so...
>
> if in_the_cache:
> return cached
> else:
> result = MyModel.all().filter('somekey =',
> x).order('the_order')
> put_in_cache(result)
> return result
>
> ...and I invalidate the cache whenever a write is made, but the values
> change often enough that the caching is essentially useless (this is a
> write-heavy app).
>
> One option I'm considering is to recalculate and cache the result
> whenever a write happens, but that could result in a lot of
> unnecessary CPU usage, because, again, this is a write-heavy app, and
> I only do about one read for every 20-30 writes.
>
> Ideally there would be a way to optimize the way I model the data or
> query the data, such that I could get the hundreds of objects in a
> shorter period of time without sacrificing CPU usage. Are there any
> fancy data modeling tricks I can use, like sharding them over multiple
> models (which doesn't seem like it would make any difference), or
> using ListProperty (in which case I might run into the limit of items
> in a ListProperty, because each MyModel record has four attributes I
> need to include), or something else...?
>
> Adrian
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected]<google-appengine%[email protected]>
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
--
Jeff
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
