On Wed, Aug 31, 2011 at 3:10 AM, pdknsk <[email protected]> wrote: > I haven't been able to figure out what the reason is. It seems to have > improved slightly to about 3/s but it still had 20 seconds delays > occasionally for no apparent reason. I've since moved mail sending to > a named version (rather than backend), and it works great. It can > easily send 6/s, with one instance.
This sounds related to a thread I started a couple weeks ago: https://groups.google.com/d/topic/google-appengine/rAmZi6a8ZaI/discussion If I understand my results correctly, every request to a backend has an extra 100+ms added to it in transit (before your code starts executing). It could actually be considerably worse; doing a urlfetch from a frontend to a backend (no-op) typically takes hundreds of milliseconds. Combined with doing actual work per request, this could explain low throughput numbers. It would make sense that the task queue is overloading the backend and therefore failing a lot of requests. These failures will cause the task queue to back off, producing the longer delays you see. This really isn't a good application for a backend. If you want to split logs out, put your processing on a separate frontend version. Alternatively, you could convert your backend push-queue to a pull-queue... but it really sounds like a job for a properly scaled and load-balanced frontend. If you are worried about spinning up too many instances, throttle the queue with max-concurrent-requests. Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
