I'm guessing there's just some weird coding you're doing that's causing the breakage.
Personally, I'd make all the cron jobs just one cron.. and have that one cron simply add one named start task to a queue... then have that start task do all of your initial task adding using named tasks with the appropriate countdown (or eta) and try,except handling TaskAlreadyExiststs and TombStonedTask errors by passing silently. A starting task like that will always succeed.. it will just keep retrying until every named task it should create is created. Then, you can get down to debugging what it is about your queues that may still be borking (I don't use rates in my queues.. so I am not familiar with gotchas in that area). On Sat, Feb 27, 2010 at 3:06 AM, Marc Provost <[email protected]> wrote: > Thanks for you help Eli! I didn't know about the unique name feature. > I will use that trick, at least for the most important crons. Still, I > wonder why I see this failure if my requests / seconds is never higher > than 3 in my dashboard. I mean, if I saw spikes close to 30 at least I > could start debugging. And the failure rate is much higher than 0.1%. > It's more like 5%. And it's weird, sometimes the first cron will fail > when there is no activity at all... why? It always fail after 10 > seconds. Sometimes the first cron fails, sometimes the 5th, etc. And > my app itself has almost no traffic. > > A few notes: > > * Yeah, I use chaining. I am not adding the 1000s tasks in one shot. > Sorry for not specifying that. None of my cron/queue is close to the > 30 second limit. I didn't see a 30 second timeout error in my logs for > a long time. > * I use a queue with a rate of 5 tasks per second, so approximately > 300 tasks will go through per minute and 1200 tasks in 4 minutes. > Hence my approximation. I also manually ran the cron jobs and saw them > complete in about 4 minutes. > > Basically, the question I'm trying to answer here is: > > Why I'm I seeing the "simultaneous dynamic request limit error" if: > > 1) In my dashboard, the max rate of requests I see is 3. Far from 30. > 2) The rate of my queue is 5 / second. Again, far from 30. > 3) This error occurs even for the first cron job, when there is no > other tasks / cron running (and my app has almost no traffic) > > Thanks again for your tips. > Marc > > > > > On Feb 26, 8:16 pm, Wooble <[email protected]> wrote: > > On Feb 26, 7:46 pm, Locke <[email protected]> wrote: > > > > > I have also seen this timeout error when trying to add to the task > > > queue. What is interesting to me is that it kills my process after 10 > > > seconds, instead of the thirty seconds we supposedly are allowed. > > > > The 30 seconds are for a request that actually runs. This message > > indicates your request handler didn't get run at all, because too many > > instances of your application were already running. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
