You could also get a little boost by making it a key's only query (by using listKeys), but since you're wanting to simulate some in-code filtering you'll beed to whole entity.
Perhaps a Java person can chime in with more ideas / insights. Robert On Mon, Feb 7, 2011 at 13:30, HalcyonDays <[email protected]> wrote: > Thanks, I'll definitely run it through appstats as soon as I get a > chance... > > ...here's the code, though, if you'd like to take a look. Nothing too > crazy. I'm using Objectify. > > > Objectify ofy = new DAO().ofy(); > ArrayList<Key<FakeEntity>> arr = new ArrayList<Key<FakeEntity>>(); > > Query<FakeEntity> query = > ofy.query(FakeEntity.class).filter("lookAtMe", "woo"); > QueryResultIterator<FakeEntity> iterator = query.iterator(); > > while(iterator.hasNext()){ > arr.add(iterator.next().getKey()); > } > > ofy.delete(arr); > > > Essentially just gets an objectify object, does a query over my test > object "FakeEntity" looking for all entries whose "lookAtMe" field is > set to "woo," iterates over them, adding to a list, and then passes > the list to the delete call. > > I know iterating over each one is not necessary with Objectify and I > can simply pass the iterator to the delete call, but my test is > simulating what the real code will need to do (i.e. look at each > object before determining if it can be deleted or not.) > > Nothing too crazy, but there may be some fundamental misunderstanding > of how bulk deletes should be done. > > > On Feb 7, 12:34 pm, Robert Kluin <[email protected]> wrote: >> Sometimes it can be more expensive than expected, but there is also a >> fair chance your code is missing some 'basic' (ie common app engine) >> optimizations. Use Appstats to profile your code, look for lots of >> RPC calls done in serial. Make sure you read about keys only queries >> and batch operations. >> >> Also, post your test code. If there is low-hanging-fruit, people will >> (probably) let you know about it pretty quickly. >> >> http://code.google.com/appengine/docs/java/tools/appstats.htmlhttp://code.google.com/appengine/docs/python/tools/appstats.html >> >> Robert >> >> >> >> On Mon, Feb 7, 2011 at 11:54, HalcyonDays <[email protected]> wrote: >> > Hello all - >> >> > I've been doing a few load tests with the task queue, and I ran across >> > some interestingly large numbers... I constructed a very simple test >> > to put batches of a thousand or so fake items into the datastore at a >> > time, and then created a task to see how many I could query, count, >> > and remove in a reasonable amount of time. Wall (real) time was >> > fantastic for both tests, roughly 18653 ms to delete 21,000 of my test >> > items... the number that had me a teensy bit worried was the CPU >> > time... 1494101 cpu_ms. So... essentially 20-some minutes. If this >> > is normal, then it's just something I'll have to be aware of for any >> > expensive tasks... but is this standard/to be expected? It seems odd, >> > even on something as massively parallel as the datastore that I would >> > be able to rack up that much CPU time in a little under 20 seconds. >> >> > Anyway, just curious. >> >> > Thanks! >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
