> > > Today I decided to go faster and configured the queue to run at 20/s with > max_concurrent of 3000,
Doesnt that explicity say that its allowed to process 3000 tasks at once? With no multi-threading that means pretty much 3000 instances. the 20/s is just the starting point - it can go higher. In effect the queue is trying to span 20 new tasks each and every second. But because tasks are talking longer than a second - it can't be run on a fixed number of instances. in the first second 20 instances are spawned. In the second second, 20 more instances, because the first 20 are still busy. In the third, 20 more. --- basically this pattern continues. Some tasks do eventually finish, freeing up the instance - but new tasks are still coming quicker than your instances can process them. And because you are loading the remote service even more, the RPCs are probably getting slower - just compuunding your problem. Lower max_concurrent right away! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
