I noticed this issue as well when working from a bulkloaded backup up my live datastore (around the same number of entities). For me, it seems that db.delete() takes forever.. db.puts() seem tolerable. (I'm on Windows).
I see that Nick is indicating that this may just be the way it is. It'd be nice to do offline processing and development working against a snapshot of my current datastore.. but I'm not going to whine too much for that. I'd rather the App Engine Team keep focusing on sweetening the live store. One potential way around this (if you realllly, really wanted to use the development environment with a large datastore).. you could cook up a way to have the datastore stuff loaded into a global Dictionary object.. and you could put(), delete() stuff from it while testing. That might work well since the development environment seems to run only one instance or thread or whatever they call it.. so as long as you didn't fill up the memory, it seems that maybe the global object would be there for long enough to do several tests.. Then again, you'd probably need uncoupled Models.. where no class was related to any other one through references or ancestors or parents or children etc. So, like I said, you'd have to really want to do this.. For me, this would work since I want to do lots of cycling through 10s of thousands of entities.. updating and deleting them in memory (Model entities are relate to others only through their key_names and not through any defined properties)... then when I was done.. I could just have some process sync the ram datastore with the one on disk.. and go have a beer or whatever while it took its sweet time. There be dragons in the details though. On Fri, Feb 19, 2010 at 12:19 PM, obvious <[email protected]> wrote: > I've loaded up a local datastore with 40,000+ entries. Unfortunately, > recalling any data from it at all is very slow on my fairly new > Macbook Pro. Any suggestions on speeding things up, short of buying a > new piece of hardware? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
