>Today I decided to go faster and configured the queue to run at 20/s with max_concurrent of 3000, s
I am surprised that if you don't have Multi-thread enabled that you don't have 3000 instances. Probably you hit a hard Quota limit. I would have borrowed Ikai's code. http://ikaisays.com/2010/06/29/using-asynchronous-urlfetch-on-java-app-engin e/ Done the fetches Then async puts to the DB. I'd have done this on a Back End Instance so the task could take longer. 150k isn't very many, Assuming the fetch completed in a reasonable amount of time I'd do 8 or 15 at a time. I'd have another app/url set the pace. You could set a cron to fire every 5s or something similar on a bit of code that all it did was fire 4 of the backends, or 8 or 16 or 64. So that you could change the speed as you saw that things were working. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
