While rapidly processing leased tasks from a pull-queue, I consistently had a problem with Transient Errors when requesting a fixed number of tasks (1000 at a time) prior to iterating through the tasks. The only time we see Transient Errors in pull queues is this lease step. There was never any discernible pattern, so I think this may be due to infrastructure load not visible to the app Dashboard. Added logic to pause for a second, then retry the getting the leased items when the lease request throws a TE exception. If after a second try, we log an error. Have never seen the notation in the error log for this second try failing, so for us this pause seems to fix it. Note that we never had large number of TEs all at once, but there were clearly times when they appeared more often than others. I think I sent some logs to a Google engineer identifying such a pattern. No conclusion from this though, so now we always use this TE sleep/retry exception handling for all pull-queue lease logic. Fortunately even then they are happening, TEs are infrequent, so 60 seconds of sleep in our big cron task has no real affect on instances etc. If you find the TEs are due to the lease step, this might help. Best, Steve
On Friday, March 27, 2015 at 10:47:32 AM UTC-7, Kaan Soral wrote: > > These past few hours I'm seeing some TransientError flurries, momentarily > the queue produces lots of TransientError's > > I'm using tags with the pull queues, I'm also using multiple queues to > reduce/distribute the load > > Is this a temporary issue, or are there hidden limits to pull queue usage > that I should adhere to? (like '*max. 10 consequent/simultaneous lease > operations'* etc.) > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9e44047c-c053-4350-ac2d-a9a11114a615%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
