Hi, I'm calling a webservice which returns an XML with 100 items. This webservice is called many times a second.
I want to save the items as fast as possible and so I was thinking of creating a task for each item (with the serialized item object as task payload). Each such task does a select on the datastore to know if the item already exists and if it doesn't exist, inserts a new entity (or else update the entity). I suppose adding a task also has overhead but still is faster than calling the datastore. I'd like to ask if any of you has considered this approach or has some performance numbers or advice about which is best (fastest): - perform all 100 selects and updates in one task (here the 30 second execution limit is dangerous) - one task for each item insert (since the queue call rate is limited to 50, the "one task per item" approach will take almost 2 seconds) - one task for each item insert where the tasks are round-robin added to 5 queues so the call rate is increased to 250 / second - 10 tasks with each 10 inserts (here the problem can become the task payload which is limited to 10 KB) Or should I ask for a queue call rate increase (don't think this is possible)? Thanks, Pieter Coucke -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
