Hi again.
> > > 2. How up to date must your view be, > > It's OK if it's a few seconds out of date, but not much longer than > that is acceptable. > That gives you some room to move > > 3. Is it feasible to refetch/populate your cache via a background > > task (if you write 20-30 times per read) > > Yes, this is something I've considered doing, but given my answer to > the previous question (reads have to be fresh within a few seconds), > that suggests the background task would have to run after each write, > or regularly every few seconds or something -- and that seems wasteful > in terms of App Engine's CPU usage. > Ok, but don't do it on every write. One approach would be, If you know you are writing constantly, create a memcache entity to say you have scheduled a task and then run the task, (say to run in two seconds). On each write check that it has gone, then schedule a new task. When the task is run the last thing it does is delete the memcache record, because it only runs every few writes/ secs, (you will need to play here) it should accumulate any intervening writes. Also give the memcache record a short life span, so that if something goes wrong, it will expire say in 20 secs. > > Also I assume you do a fetch on the query rather than iterating of the > > result (fetch is much quicker). > > Oh, we could be onto something here. I'm iterating over the result, > not using fetch, because I was under the impression fetch() was only > for a limited number of records. Are you saying that I could pass a > dummy huge number to fetch(), like fetch(1000000), and it would be > faster than iterating? If so, that would be an easy win. > You can only do a fetch of max 1000 items. You can perform queries (using cursors) that have more than 1000 results. What I think your background tasks should do is perform a series of key only queries and then squirrel away in memcache the keys of the items into memcache. The you may do a few memcache gets to retrieve the full set of keys followed by db.get(a list of keys) However I don't think this won't scale to 10000's of entities. > > I think if you adding/removing keys you could actually keep a list in > > of keys in memcache as a single entity > > then fetch the list of keys then do a db.get(list_of_keys) assuming > > you have less 1000 entities as you are still limited to 1000's items > > retrieved. > > This is a good idea, too -- thanks. I'll give it a shot. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
