Following are the answers to your questions: 1. Rate: Both maximum rate and enforced rate were changed to 5 QPS. I was only using default queue and set 20 QPS on it. 2. Task: All my tasks run on frontends 3. Admin Log: Yes, I am the only developer uploading changes and there is no activity in admin log when the setting was changed.
In addition, when I tried to change the setting to 100 QPS this morning in order to drain the queue, it was always changed back to 5 QPS about 5 minutes later after the deployment. Finally I found a workaround to create another queue with the same setting which stopped the madness of automatically configuration change. Actually task queue has been causing service interruption to me in the past 2 days. Yesterday, the default queue suddenly stopped accepting any task with name, which I had to workaround in order to bring my service back to normal state. I was too tired to write a post here about the issue after I finished live site support yesterday. Since you are you investigating, would you please also look into this issue as well? I really need the feature to add task with name so that I can avoid duplicated task processing. On Thursday, October 18, 2012 7:11:39 PM UTC-7, nverne wrote: > > Could you please give us some more details: > > Which rate did you see go down? The Maximum Rate or the Enforced Rate? > > Were your tasks running on backends or frontends? > > Do your admin logs indicate that you were the only developer uploading > changes? > > On Fri, Oct 19, 2012 at 12:59 PM, nischalshetty > <[email protected]<javascript:> > > wrote: > >> We are facing problems with the task queue as well. The tasks shot up to >> millions and everything stopped working for us! >> >> >> On Thursday, October 18, 2012 11:34:23 PM UTC+5:30, gks wrote: >>> >>> And the task queue of my app aos-prod-1 is lying about the # of tasks in >>> queue since it always says 100, which is impossible given my customers >>> usage pattern. >>> >>> On Thursday, October 18, 2012 10:19:09 AM UTC-7, gks wrote: >>>> >>>> After I changed the processing rate to 100QPS to drain the queue, it >>>> was changed back to 5QPS just now. Can anyone from Google start looking >>>> into this issue and stop changing my task queue setting? >>>> >>>> On Thursday, October 18, 2012 9:42:17 AM UTC-7, gks wrote: >>>>> >>>>> I just found task queue processing rate of my apps was changed to 5qps >>>>> last night without any deployment from me! It caused a major service >>>>> interruption to my customers since a lot of tasks were delayed. >>>>> >>>>> There is no way this change came from my team since everyone was >>>>> sleeping when the change happened. And there is no corresponding >>>>> deployment >>>>> activity from admin log. Can Google provide an explanation on why it >>>>> happened, and what can be done to prevent it from happening again? >>>>> >>>>> My AppID are: aos-prod-1, aos-prod-2 >>>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/google-appengine/-/SIo8pCc0-WYJ. >> >> To post to this group, send email to >> [email protected]<javascript:> >> . >> To unsubscribe from this group, send email to >> [email protected] <javascript:>. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/vgNw6t7sfeYJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
