I would you mapreduce for GAE, see http://code.google.com/p/appengine-mapreduce/. It has been integrated with latest SDK so no need to download, I use it with Python--just make sure to import google.appengine.ext.mapreduce.
On 29 lis, 10:06, dflorey <[email protected]> wrote: > Hi, > I'm looking for the most effective way to update 50000 entities + one > of the child entities each. > Right now I'm using a task per transaction to be able to modify the > entity and the child entities inside a transaction to make the task > idempotent. > I'm using sharded counters to check when the operation is done. > Everything works fine, but it takes very long (=minutes to hours) to > perform the modifications. > I'm getting no concurrent modification exceptions etc. at all, but > tasks get rescheduled for some reason and wait for a long time before > getting executed depending on the number of retries. > > Is there a way to speed things up? > I'm looking for a solution that will execute the update almost > immediately :-) > My tasks take less than 1000ms each and I can see ~30 instances in the > dashboard. > > Thanks for any ideas, > > Daniel -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
