The datastore isn't a relational database, and creating an "index" isn't the same as it is in SQL - you aren't creating a B-tree index that allows a fast query, you're creating an actual index so the datastore knows which entities have the value you are looking for. Basically, this index has been created already.
8 seconds is pretty high, but in general, it depends on the size of your entities and whether offset is being used. The datastore is a distributed key-value store. Rather than query for 1000 entities, is there a way you can limit what you are querying for to a key-value retrieval? Think of the datastore as a giant distributed, persistent hash table with additional functionality: how would you optimize data retrieval for this type of store? My guess is the functionality you're looking for can be either cached, or you can do a keys only query and lazy query, or you can try to store the data in sharded "slabs" - that is, store a denormalized cache of the data you need from 200 temporally sharded lists of posts - it's unlikely that you need all the data from each post. A good project to study would be the Jaiku engine: http://code.google.com/p/jaikuengine/ On Tue, Feb 9, 2010 at 9:10 PM, Brian <[email protected]> wrote: > I am also getting and closing a new PersistenceManager around the > query, if that matters > > On Feb 9, 11:08 pm, Brian <[email protected]> wrote: > > Hi guys, > > > > I was wondering if anyone has found a less CPU intensive way to query. > > I basically have a bunch of small objects in the database I want out. > > I tried a few ways to get them out, including an extent and a query. I > > ended up settling on this: > > > > Query query = pm.newQuery(Post.class); > > query.setOrdering("postedDate desc"); > > query.setRange(start, end); > > List<Post> posts = (List<Post>) query.execute(); > > > > Doing a range of 0 to 1000.. figuring the 1000 newest items is good > > enough. > > > > I find that this tends to be pretty bad on my quota CPU.. like maybe 8 > > seconds of CPU time per query, if there are 1000 posts. Is this just > > how it is? Or do I need to add an index to postedDate maybe? > > > > I plan to play with it more tomorow, but if anyone has insight on > > getting stuff out of the database fast (from both a time and CPU quota > > standpoint), I would appreciate it. I have never seen a datbase take> .1 > seconds for such a stupid silly query, I have to be doing > > > > something wrong. > > > > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine-java%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
