That's not what I was asking, though thanks for the information. It appears that my task queues that feed the backends are backing up (because there are not enough instances, by configuration), so I'm guessing that's my answer.
j On Sep 13, 1:38 pm, Rishi Arora <[email protected]> wrote: > Check this > out:http://code.google.com/appengine/docs/python/taskqueue/overview-push.... > > This suggests that tasks pushed onto taskqueues must get executed within 10 > minutes, in the case of front-end instances. But backends are exempt from > this limit. This implies that when you call > tasqueue.add(taskqueue.Task(url=[xyz], target=[my backend name])), then you > have more than 10 minutes to execute the task. I don't know the exact > deadline, but it seems safe to assume it is a lot longer than 10 minutes. > > Also, one alternative for driving your backends is through cron jobs and > pull queues. This is how we recently re-designed our app to try to lower > our billable instance hours. We have one backend configured, with num > instances set to 1. This means we will never have more than 1 backend > instances. Any tasks that we want to have this backend process, we enqueue > them to a "pull queue". This enqueue operation does not trigger the backend. > We trigger the backend once every hour using a cron job. When the backend > wakes up form this trigger, it starts calling "lease_tasks" method on the > pull queue to start dequeueing pending requests one at a time. When all > requests are processed serially, the backend goes idle. The GAE scheduler > will wait around 5 minutes to stop this backend. But you'll be billed for > 15 minutes after the backend goes idle. Of course this design assumes that > the kind of requests we enqueue to the pull queue don't require immediate > processing. In our case, we can tolerate an hour of delay before these > requests are processed. But these are the kinds of things backends are most > suitable for anyways. > > Hope this helps. > > Rishi > > On Tue, Sep 13, 2011 at 12:28 PM, Jason Collins > <[email protected]>wrote: > > > > > > > > > We are moving much of our taskqueue work to dynamic backends. > > > One obvious question we're faced with is "how many (max) instances do > > we need for our background work?" > > > If we are feeding all of our work to our dynamic backends via > > taskqueue, will we see the queues get backed up if the backend > > instances cannot keep up? > > > Or, alternatively, do the queued tasks pop off at their configured > > rate and drop into a different request queue for the backend pool? > > > If the latter, how long will the requests stay on this other "backend > > pool request queue" (e.g., they will stay for 10s on the front-end > > instance request queue, or have in the past; is there an equivalent > > timeout for backends)? Is there any way to get visibility into this > > other queue, if it exists? > > > Thanks for any info, > > j > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
