Task Queues coming out of labs is great news! Will performance and stability improve at this time as well? Looking at the Taskqueue Status page, every single day for the last 3 months there have been huge spikes in execution latency. Every single day (I clicked through them all). So much yellow and red in the graphs. The last time there was a solid blue graph was July 11, 2010.
http://code.google.com/status/appengine/detail/taskqueue/2010/10/15#ae-trust-detail-taskqueue-execution-latency I realize Task Queues are for background processing which is how I use them in my apps. However, when I set the rate at 50 tasks per second I expect actual throughput to be somewhat close to that rate. When execution latency jumps up to 25 seconds per task, then essentially all the benefits of using task queues is lost. Can we expect improvements in this area when the "experimental" tag is removed? -Cameron On Oct 13, 4:42 pm, "Ikai Lan (Google)" <[email protected]> wrote: > Hey everyone, > > We're working on moving Task Queues out of labs/experimental for an upcoming > release slated for November. One of the requirements for doing so is to > better enforce the Task Queues storage quota. The documentation describes > this quota as storage for data for tasks that have not yet executed: > > http://code.google.com/appengine/docs/quotas.html#Task_Queue > > Currently, this quota is not enforced, and it is possible for applications > with backed up task queues to exceed this quota without running into quota > denials. Check your admin console's quota page for your application to see > if you are reaching or exceeding this quota. If so, you'll have the > following options: > > 1. Allocate more storage quota to taskqueue storage by updating: > - total_storage_limit in queue.yaml for Python applications > - TotalStorageLimit in queue.xml for Java applications. > Note: you may need to buy more disk quota if there's not enough > quota for datastore, blobstore and taskqueue storage altogether. > > 2. Fix backed up queues using one of the options below. You will recognize > these as queues with high numbers of tasks (typically >2000). > - Purge backed up queues (an easy button click in admin console's > Queue Details page for the queue). > - Increase the execution rate and let your app work through the > backlog. To do this, edit queue.yaml (or queue.xml for Java apps) and alter > the rates on the queues. You can set a higher bucket size than the default > (currently 5) to maximize throughput. > - Reduce the rate at which new tasks are added to queues. This will > involve some editing of app code. > > It can take a few hours to update the admin console after the quotas have > been updated. > > We'll post a reminders to the App Engine blog as well as to this thread as > the release nears. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
